A Python Book Beginning Python, Advanced Python, and

APythonBook:BeginningPython,AdvancedPython,and
PythonExercises
Author:
DaveKuhlman
Address:
dkuhlman@rexx.comhttp://www.rexx.com/~dkuhlman
Page1
Revision
1.1a
Date
April22,2012
Copyright
Copyright(c)2009DaveKuhlman.AllRightsReserved.Thisdocumentissubject
totheprovisionsoftheOpenSourceMITLicense
http://www.opensource.org/licenses/mitlicense.php.
Abstract
ThisdocumentisaselflearningdocumentforacourseinPythonprogramming.
Thiscoursecontains(1)apartforbeginners,(2)adiscussionofseveraladvanced
topicsthatareofinteresttoPythonprogrammers,and(3)aPythonworkbookwith
lotsofexercises.
Page2
Contents
1Part1BeginningPython.............................................................................................9
1.1IntroductionPython101BeginningPython....................................................9
1.1.1ImportantFeaturesofPython..........................................................................9
1.1.2WheretoGoForAdditionalhelp..................................................................10
1.2InteractivePython.................................................................................................10
1.3Lexicalmatters......................................................................................................11
1.3.1Lines..............................................................................................................11
1.3.2NamesandTokens........................................................................................11
1.3.3BlocksandIndentation..................................................................................12
1.3.4DocStrings....................................................................................................12
1.3.5Operators.......................................................................................................12
1.3.6AlsoSee.........................................................................................................14
1.3.7CodeEvaluation............................................................................................14
1.4BuiltinDataTypes...............................................................................................14
1.4.1Strings............................................................................................................14
1.4.1.1Whatstringsare.....................................................................................14
1.4.1.2Whentousestrings................................................................................15
1.4.1.3Howtousestrings.................................................................................15
1.4.2SequencesListsandTuples.......................................................................18
1.4.2.1Whatsequencesare................................................................................18
1.4.2.2Whentousesequences..........................................................................18
1.4.2.3Howtousesequences............................................................................18
1.4.3Dictionaries....................................................................................................20
1.4.3.1Whatdictionariesare.............................................................................20
1.4.3.2Whentousedictionaries........................................................................21
1.4.3.3Howtousedictionaries.........................................................................21
1.4.4Files...............................................................................................................24
1.4.4.1Whatfilesare.........................................................................................24
1.4.4.2Whentousefiles...................................................................................24
1.4.4.3Howtousefiles.....................................................................................24
1.4.4.4ReadingTextFiles.................................................................................25
1.5SimpleStatements.................................................................................................26
1.5.1printstatement...............................................................................................26
1.5.2Assignmentstatement....................................................................................27
1.5.3importstatement............................................................................................28
1.5.4assertstatement..............................................................................................29
1.5.5globalstatement.............................................................................................30
1.6CompoundstatmentsControlStructures...........................................................31
1.6.1if:statement...................................................................................................31
Page3
1.6.2for:statement.................................................................................................33
1.6.2.1Thefor:statementandunpacking..........................................................35
1.6.3while:statement.............................................................................................35
1.6.4try:except:andraiseExceptions................................................................37
1.7Organization..........................................................................................................38
1.7.1Functions.......................................................................................................39
1.7.1.1Abasicfunction.....................................................................................39
1.7.1.2Afunctionwithdefaultarguments........................................................39
1.7.1.3Argumentlistsandkeywordargumentlists..........................................39
1.7.1.4Callingafunctionwithkeywordarguments..........................................40
1.7.2Classesandinstances.....................................................................................41
1.7.2.1Abasicclass..........................................................................................41
1.7.2.2Inheritance.............................................................................................42
1.7.2.3Classdata...............................................................................................43
1.7.2.4Staticmethodsandclassmethods..........................................................43
1.7.2.5Properties...............................................................................................45
1.7.3Modules.........................................................................................................46
1.7.4Packages........................................................................................................48
1.8AcknowledgementsandThanks...........................................................................50
1.9SeeAlso................................................................................................................50
2Part2AdvancedPython............................................................................................51
2.1IntroductionPython201(Slightly)AdvancedPythonTopics.......................51
2.2RegularExpressions..............................................................................................51
2.2.1Definingregularexpressions.........................................................................51
2.2.2Compilingregularexpressions......................................................................52
2.2.3Usingregularexpressions..............................................................................52
2.2.4Usingmatchobjectstoextractavalue..........................................................53
2.2.5Extractingmultipleitems..............................................................................54
2.2.6Replacingmultipleitems...............................................................................55
2.3IteratorObjects......................................................................................................57
2.3.1ExampleAgeneratorfunction....................................................................59
2.3.2ExampleAclasscontainingageneratormethod........................................61
2.3.3ExampleAniteratorclass...........................................................................62
2.3.4ExampleAniteratorclassthatusesyield...................................................64
2.3.5ExampleAlistcomprehension...................................................................65
2.3.6ExampleAgeneratorexpression................................................................66
2.4UnitTests..............................................................................................................66
2.4.1Definingunittests..........................................................................................67
2.4.1.1Createatestclass...................................................................................67
2.5ExtendingandembeddingPython........................................................................69
2.5.1Introductionandconcepts..............................................................................69
2.5.2Extensionmodules.........................................................................................70
Page4
2.5.3SWIG.............................................................................................................72
2.5.4Pyrex..............................................................................................................75
2.5.5SWIGvs.Pyrex.............................................................................................79
2.5.6Cython...........................................................................................................79
2.5.7Extensiontypes..............................................................................................81
2.5.8Extensionclasses...........................................................................................81
2.6Parsing...................................................................................................................81
2.6.1Specialpurposeparsers.................................................................................82
2.6.2Writingarecursivedescentparserbyhand...................................................82
2.6.3Creatingalexer/tokenizerwithPlex.............................................................90
2.6.4Asurveyofexistingtools..............................................................................99
2.6.5CreatingaparserwithPLY...........................................................................99
2.6.6Creatingaparserwithpyparsing.................................................................105
2.6.6.1Parsingcommadelimitedlines............................................................106
2.6.6.2Parsingfunctors...................................................................................107
2.6.6.3Parsingnames,phonenumbers,etc.....................................................108
2.6.6.4Amorecomplexexample....................................................................109
2.7GUIApplications................................................................................................110
2.7.1Introduction.................................................................................................110
2.7.2PyGtk...........................................................................................................111
2.7.2.1Asimplemessagedialogbox..............................................................111
2.7.2.2Asimpletextinputdialogbox.............................................................113
2.7.2.3Afileselectiondialogbox...................................................................115
2.7.3EasyGUI......................................................................................................117
2.7.3.1AsimpleEasyGUIexample................................................................118
2.7.3.2AnEasyGUIfileopendialogexample................................................118
2.8GuidanceonPackagesandModules...................................................................118
2.8.1Introduction.................................................................................................118
2.8.2ImplementingPackages...............................................................................118
2.8.3UsingPackages............................................................................................119
2.8.4DistributingandInstallingPackages...........................................................119
2.9EndMatter...........................................................................................................121
2.9.1AcknowledgementsandThanks..................................................................121
2.9.2SeeAlso.......................................................................................................121
3Part3PythonWorkbook.........................................................................................122
3.1Introduction.........................................................................................................122
3.2LexicalStructures................................................................................................122
3.2.1Variablesandnames....................................................................................122
3.2.2Linestructure...............................................................................................124
3.2.3Indentationandprogramstructure...............................................................125
3.3ExecutionModel.................................................................................................126
3.4BuiltinDataTypes.............................................................................................126
Page5
3.4.1Numbers......................................................................................................127
3.4.1.1Literalrepresentationsofnumbers......................................................127
3.4.1.2Operatorsfornumbers.........................................................................129
3.4.1.3Methodsonnumbers............................................................................131
3.4.2Lists.............................................................................................................131
3.4.2.1Literalrepresentationoflists...............................................................132
3.4.2.2Operatorsonlists.................................................................................133
3.4.2.3Methodsonlists...................................................................................134
3.4.2.4Listcomprehensions............................................................................135
3.4.3Strings..........................................................................................................137
3.4.3.1Characters............................................................................................138
3.4.3.2Operatorsonstrings.............................................................................139
3.4.3.3Methodsonstrings...............................................................................140
3.4.3.4Rawstrings..........................................................................................142
3.4.3.5Unicodestrings....................................................................................143
3.4.4Dictionaries..................................................................................................144
3.4.4.1Literalrepresentationofdictionaries...................................................144
3.4.4.2Operatorsondictionaries.....................................................................145
3.4.4.3Methodsondictionaries.......................................................................146
3.4.5Files.............................................................................................................149
3.4.6Afewmiscellaneousdatatypes..................................................................151
3.4.6.1None.....................................................................................................151
3.4.6.2ThebooleansTrueandFalse...............................................................151
3.5Statements...........................................................................................................152
3.5.1Assignmentstatement..................................................................................152
3.5.2printstatement.............................................................................................154
3.5.3if:statementexercises..................................................................................154
3.5.4for:statementexercises...............................................................................155
3.5.5while:statementexercises...........................................................................158
3.5.6breakandcontinuestatements.....................................................................159
3.5.7Exceptionsandthetry:except:andraisestatements...................................160
3.6Functions.............................................................................................................162
3.6.1Optionalargumentsanddefaultvalues.......................................................163
3.6.2Passingfunctionsasarguments...................................................................165
3.6.3Extraargsandkeywordargs.......................................................................166
3.6.3.1Orderofarguments(positional,extra,andkeywordargs)..................168
3.6.4Functionsandducktypingandpolymorphism...........................................168
3.6.5Recursivefunctions.....................................................................................170
3.6.6Generatorsanditerators...............................................................................171
3.7Objectorientedprogrammingandclasses..........................................................174
3.7.1Theconstructor............................................................................................175
3.7.2InheritanceImplementingasubclass.......................................................177
Page6
3.7.3Classesandpolymorphism..........................................................................179
3.7.4Recursivecallstomethods..........................................................................180
3.7.5Classvariables,classmethods,andstaticmethods.....................................181
3.7.5.1Decoratorsforclassmethodandstaticmethod.....................................184
3.8AdditionalandAdvancedTopics........................................................................185
3.8.1Decoratorsandhowtoimplementthem......................................................185
3.8.1.1Decoratorswitharguments..................................................................186
3.8.1.2Stackeddecorators...............................................................................187
3.8.1.3Morehelpwithdecorators...................................................................189
3.8.2Iterables.......................................................................................................190
3.8.2.1AfewpreliminariesonIterables..........................................................190
3.8.2.2Morehelpwithiterables......................................................................191
3.9ApplicationsandRecipes....................................................................................191
3.9.1XMLSAX,minidom,ElementTree,Lxml..............................................191
3.9.2Relationaldatabaseaccess...........................................................................199
3.9.3CSVcommaseparatedvaluefiles...........................................................205
3.9.4YAMLandPyYAML..................................................................................206
3.9.5Json..............................................................................................................207
4Part4GeneratingPythonBindingsforXML.........................................................210
4.1Introduction.........................................................................................................210
4.2Generatingthecode.............................................................................................211
4.3UsingthegeneratedcodetoparseandexportanXMLdocument.....................213
4.4Somecommandlineoptionsyoumightwanttoknow.......................................213
4.5Thegraphicalfrontend.......................................................................................214
4.6Addingapplicationspecificbehavior.................................................................214
4.6.1Implementingcustomsubclasses................................................................215
4.6.2Usingthegenerated"API"fromyourapplication......................................215
4.6.3Acombinedapproach..................................................................................216
4.7Specialsituationsanduses..................................................................................218
4.7.1Generic,typeindependentprocessing.........................................................218
4.7.1.1Step1generatethebindings............................................................219
4.7.1.2Step2addapplicationspecificcode................................................220
4.7.1.3Step3writeatest/driverharness.....................................................224
4.7.1.4Step4runthetestapplication..........................................................225
4.8Somehints...........................................................................................................225
4.8.1ChildrendefinedwithmaxOccursgreaterthan1........................................225
4.8.2Childrendefinedwithsimplenumerictypes...............................................226
4.8.3Thetypeofanelement'scharactercontent..................................................226
4.8.4Constructorsandtheirdefaultvalues..........................................................226
Page7
Preface
ThisbookisacollectionofmaterialsthatI'veusedwhenconductingPythontrainingand
alsomaterialsfrommyWebsitethatareintendedforselfinstruction.
Youmaypreferamachinereadablecopyofthisbook.Youcanfinditinvariousformats
here:
HTMLhttp://www.rexx.com/~dkuhlman/python_book_01.html
PDFhttp://www.rexx.com/~dkuhlman/python_book_01.pdf
ODF/OpenOfficehttp://www.rexx.com/~dkuhlman/python_book_01.odt
And,letmethankthestudentsinmyPythonclasses.Theirquestionsandsuggestions
wereagreathelpinthepreparationofthesematerials.
Page8
1Part1BeginningPython
1.1IntroductionPython101BeginningPython
Pythonisahighlevelgeneralpurposeprogramminglanguage:
Becausecodeisautomaticallycompiledtobytecodeandexecuted,Pythonis
suitableforuseasascriptinglanguage,Webapplicationimplementation
language,etc.
BecausePythoncanbeextendedinCandC++,Pythoncanprovidethespeed
neededforevencomputeintensivetasks.
Becauseofitsstrongstructuringconstructs(nestedcodeblocks,functions,
classes,modules,andpackages)anditsconsistentuseofobjectsandobject
orientedprogramming,Pythonenablesustowriteclear,logicalapplicationsfor
smallandlargetasks.
1.1.1ImportantFeaturesofPython
Builtinhighleveldatatypes:strings,lists,dictionaries,etc.
Theusualcontrolstructures:if,ifelse,ifelifelse,while,plusapowerful
collectioniterator(for).
Multiplelevelsoforganizationalstructure:functions,classes,modules,and
packages.Theseassistinorganizingcode.Anexcellentandlargeexampleisthe
Pythonstandardlibrary.
CompileontheflytobytecodeSourcecodeiscompiledtobytecodewithouta
separatecompilestep.Sourcecodemodulescanalsobe"precompiled"tobyte
codefiles.
ObjectorientedPythonprovidesaconsistentwaytouseobjects:everythingis
anobject.And,inPythonitiseasytoimplementnewobjecttypes(calledclasses
inobjectorientedprogramming).
ExtensionsinCandC++Extensionmodulesandextensiontypescanbewritten
byhand.Therearealsotoolsthathelpwiththis,forexample,SWIG,sip,Pyrex.
JythonisaversionofPythonthat"playswellwith"Java.See:TheJythonProject
http://www.jython.org/Project/.
Somethingsyouwillneedtoknow:
Pythonusesindentationtoshowblockstructure.Indentoneleveltoshowthe
beginningofablock.Outdentoneleveltoshowtheendofablock.Asan
example,thefollowingCstylecode:
if(x)
{
if(y)
Page9
{
f1()
}
f2()
}
inPythonwouldbe:
ifx:
ify:
f1()
f2()
And,theconventionistousefourspaces(andnohardtabs)foreachlevelofindentation.
Actually,it'smorethanaconvention;it'spracticallyarequirement.Followingthat
"convention"willmakeitsomucheasiertomergeyourPythoncodewithcodefrom
othersources.
1.1.2WheretoGoForAdditionalhelp
ThestandardPythondocumentationsetItcontainsatutorial,alanguage
reference,thestandardlibraryreference,anddocumentsonextendingPythonin
C/C++.Youcanfindithere:http://www.python.org/doc/.
OtherPythontutorialsSeeespecially:
Beginner'sGuidetoPythonhttp://wiki.python.org/moin/BeginnersGuide
OtherPythonresourcesSeeespecially:
Pythondocumentationhttp://www.python.org/doc/
ThePythonhomeWebsitehttp://www.python.org/
ThewholePythonFAQhttp://www.python.org/doc/FAQ.html
1.2InteractivePython
IfyouexecutePythonfromthecommandlinewithnoscript(noarguments),Python
givesyouaninteractiveprompt.ThisisanexcellentfacilityforlearningPythonandfor
tryingsmallsnippetsofcode.Manyoftheexamplesthatfollowweredevelopedusing
thePythoninteractiveprompt.
StartthePythoninteractiveinterpreterbytypingpythonwithnoargumentsatthe
commandline.Forexample:
$python
Python2.6.1(r261:67515,Jan112009,15:19:23)
[GCC4.3.2]onlinux2
Type"help","copyright","credits"or"license"formore
information.
>>>print'hello'
hello
>>>
YoumayalsowanttoconsiderusingIDLE.IDLEisagraphicalintegrateddevelopment
environmentforPython;itcontainsaPythonshell.ItislikelythatIdlewasinstalledfor
Page10
youwhenyouinstalledPython.YouwillfindascripttostartupIDLEinthe
Tools/scriptsdirectoryofyourPythondistribution.IDLErequiresTkinter.
Inaddition,therearetoolsthatwillgiveyouamorepowerfulandfancyPython
interactiveinterpreter.OneexampleisIPython,whichisavailableat
http://ipython.scipy.org/.
1.3Lexicalmatters
1.3.1Lines
Pythondoeswhatyouwantittodomostofthetimesothatyouonlyhavetoadd
extracharacterssomeofthetime.
Statementseparatorisasemicolon,butisonlyneededwhenthereismorethan
onestatementonaline.
ContinuationlinesUseabackslashattheendoftheline.But,notethatan
openingbracket(orparenthesis)makethebackslashunnecessary.
CommentsEverythingafter"#"onalineisignored.Noblockcomments,but
docstringsareacommentinquotesatthebeginningofamodule,class,method
orfunction.Also,editorswithsupportforPythonwillcommentoutaselected
blockofcode,usuallywith"##".
1.3.2NamesandTokens
Allowedcharactersinaname:azAZ09underscore,andmustbeginwitha
letterorunderscore.
Namesandidentifiersarecasesensitive.
Identifierscanbeofunlimitedlength.
Specialnames,customizing,etc.Usuallybeginandendindoubleunderscores.
SpecialnameclassesSingleanddoubleunderscores.
LeadingdoubleunderscoresNamemanglingformethodnames.
LeadingsingleunderscoreSuggestsa"private"methodnameinaclass.Not
importedby"frommoduleimport*".
TrailingsingleunderscoreSometimesusedtoavoidaconflictwitha
keyword,forexample,class_.
NamingconventionsNotrigid,buthereisonesetofrecommendations:
Modulesandpackagesalllowercase.
GlobalsandconstantsUppercase.
ClassnamesBumpycapswithinitialupper.
MethodandfunctionnamesAlllowercasewithwordsseparatedby
underscores.
LocalvariablesLowercase(possiblywithunderscorebetweenwords)or
bumpycapswithinitialloweroryourchoice.
Page11
Names/variablesinPythondonothaveatype.Valueshavetypes.
1.3.3BlocksandIndentation
Pythonrepresentsblockstructureandnestedblockstructurewithindentation,notwith
beginandendbrackets.
TheemptyblockUsethepassnoopstatement.
Benefitsoftheuseofindentationtoindicatestructure:
Reducestheneedforacodingstandard.Onlyneedtospecifythatindentationis4
spacesandnohardtabs.
Reducesinconsistency.Codefromdifferentsourcesfollowthesameindentation
style.Ithasto.
Reduceswork.Onlyneedtogettheindentationcorrect,notbothindentationand
brackets.
Reducesclutter.Eliminatesallthecurlybrackets.
Ifitlookscorrect,itiscorrect.Indentationcannotfoolthereader.
EditorconsiderationsThestandardforindentingPythoncodeis4spaces(nohardtabs)
foreachindentationlevel.Youwillneedatexteditorthathelpsyourespectthat.
1.3.4DocStrings
Docstringsarelikecomments,buttheyarecarriedwithexecutingcode.Docstringscan
beviewedwithseveraltools,e.g.help(),obj.__doc__,and,inIPython,aquestion
mark(?)afteranamewillproducehelp.
Adocstringisaquotedstringatthebeginningofamodule,function,class,ormethod.
Wecanusetriplequotingtocreatedocstringsthatspanmultiplelines.
Therearealsotoolsthatextractandformatdocstrings,forexample:
pydocDocumentationgeneratorandonlinehelpsystem.Seepydoc
http://docs.python.org/lib/modulepydoc.html.
epydocAutomaticAPIDocumentationGenerationforPython.SeeEpydoc
http://epydoc.sourceforge.net/index.html
SphinxSphinxisapowerfultoolforgeneratingPythondocumentation.See:
SphinxPythonDocumentationGeneratorhttp://sphinx.pocoo.org/.
1.3.5Operators
See:http://docs.python.org/ref/operators.html.Pythondefinesthefollowing
operators:
+***///%
<<>>&|^~
<><=>===!=<>
Page12
Thecomparisonoperators<>and!=arealternatespellingsofthesameoperator.
!=isthepreferredspelling;<>isobsolescent.
Logicaloperators:
andorisnotin
Therearealso(1)thedotoperator,(2)thesubscriptoperator[],andthe
function/methodcalloperator().
Forinformationontheprecedencesofoperators,seeSummaryofoperators
http://docs.python.org/ref/summary.html,whichisreproducedbelow.
ThefollowingtablesummarizestheoperatorprecedencesinPython,fromlowest
precedence(leastbinding)tohighestprecedence(mostbinding).Operatorsonthe
samelinehavethesameprecedence.Unlessthesyntaxisexplicitlygiven,
operatorsarebinary.Operatorsonthesamelinegrouplefttoright(exceptfor
comparisons,includingtests,whichallhavethesameprecedenceandchainfrom
lefttorightseesection5.9andexponentiation,whichgroupsfromrightto
left):
OperatorDescription
==========================================
lambdaLambdaexpression
orBooleanOR
andBooleanAND
notxBooleanNOT
in,notinMembershiptests
is,isnotIdentitytests
<,<=,>,>=,<>,!=,==Comparisons
|BitwiseOR
^BitwiseXOR
&BitwiseAND
<<,>>Shifts
+,Additionandsubtraction
*,/,%Multiplication,division,
remainder
+x,xPositive,negative
~xBitwisenot
**Exponentiation
x.attributeAttributereference
x[index]Subscription
x[index:index]Slicing
f(arguments...)Functioncall
(expressions...)Bindingortupledisplay
[expressions...]Listdisplay
{key:datum...}Dictionarydisplay
èxpressions...`Stringconversion
Notethatmostoperatorsresultincallstomethodswithspecialnames,for
example__add__,__sub__,__mul__,etc.SeeSpecialmethodnames
http://docs.python.org/ref/specialnames.html
Later,wewillseehowtheseoperatorscanbeemulatedinclassesthatyoudefine
yourself,throughtheuseofthesespecialnames.
Page13
1.3.6AlsoSee
FormoreonlexicalmattersandPythonstyles,see:
CodeLikeaPythonista:IdiomaticPython
http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html.
StyleGuideforPythonCodehttp://www.python.org/dev/peps/pep0008/
1.3.7CodeEvaluation
UnderstandingthePythonexecutionmodelHowPythonevaluatesandexecutesyour
code.Pythonevaluatesascriptormodulefromthetoptothebottom,bindingvalues
(objects)tonamesasitproceeds.
EvaluatingexpressionsExpressionsareevaluatedinkeepingwiththerulesdescribed
foroperators,above.
Creatingnames/variablesBindingThefollowingallcreatenames(variables)and
bindvalues(objects)tothem:(1)assignment,(2)functiondefinition,(3)classdefinition,
(4)functionandmethodcall,(5)importingamodule,...
FirstclassobjectsAlmostallobjectsinPythonarefirstclass.Definition:Anobjectis
firstclassif:(1)wecanputitinastructuredobject;(2)wecanpassittoafunction;and
(3)wecanreturnitfromafunction.
ReferencesObjects(orreferencestothem)canbeshared.Whatdoesthismean?
Theobject(s)satisfytheidentitytestoperatoris,thatis,obj1isobj2
returnsTrue.
Thebuiltinfunctionid(obj)returnsthesamevalue,thatis,id(obj1)==
id(obj2)isTrue.
Theconsequencesformutableobjectsaredifferentfromthoseforimmutable
objects.
Changing(updating)amutableobjectreferencedthroughonevariableor
containeralsochangesthatobjectreferencedthroughothervariablesor
containers,becauseitisthesameobject.
del()Thebuiltinfunctiondel()removesareference,not(necessarily)the
objectitself.
1.4BuiltinDataTypes
1.4.1Strings
1.4.1.1Whatstringsare
InPython,stringsareimmutablesequencesofcharacters.Theyareimmutableinthatin
Page14
ordertomodifyastring,youmustproduceanewstring.
1.4.1.2Whentousestrings
Anytextinformation.
1.4.1.3Howtousestrings
Createanewstringfromaconstant:
s1='abce'
s2="xyz"
s3="""A
multiline
string.
"""
Useanyofthestringmethods,forexample:
>>>'Thehappycatranhome.'.upper()
'THEHAPPYCATRANHOME.'
>>>'Thehappycatranhome.'.find('cat')
10
>>>'Thehappycatranhome.'.find('kitten')
1
>>>'Thehappycatranhome.'.replace('cat','dog')
'Thehappydogranhome.'
Type"help(str)"orseehttp://www.python.org/doc/current/lib/stringmethods.htmlfor
moreinformationonstringmethods.
Youcanalsousetheequivalentfunctionsfromthestringmodule.Forexample:
>>>importstring
>>>s1='Thehappycatranhome.'
>>>string.find(s1,'happy')
4
SeestringCommonstringoperationshttp://www.python.org/doc/current/lib/module
string.htmlformoreinformationonthestringmodule.
Thereisalsoastringformattingoperator:"%".Forexample:
>>>state='California'
>>>'Itneverrainsinsunny%s.'%state
'ItneverrainsinsunnyCalifornia.'
>>>
>>>width=24
>>>height=32
>>>depth=8
>>>print'Theboxis%dby%dby%d.'%(width,height,depth,)
Theboxis24by32by8.
Thingstoknow:
Page15
Formatspecifiersconsistofapercentsignfollowedbyflags,length,andatype
character.
Thenumberofformatspecifiersinthetargetstring(totheleftofthe"%"
operator)mustbethesameasthenumberofvaluesontheright.
Whentherearemorethanonevalue(ontheright),theymustbeprovidedina
tuple.
Youcanlearnaboutthevariousconversioncharactersandflagsusedtocontrolstring
formattinghere:StringFormattingOperations
http://docs.python.org/library/stdtypes.html#stringformattingoperations.
Youcanalsowritestringstoafileandreadthemfromafile.Herearesomeexamples.
WritingForexample:
>>>outfile=open('tmp.txt','w')
>>>outfile.write('Thisisline#1\n')
>>>outfile.close()
Notes:
Notetheendoflinecharacterattheendofeachstring.
Theopen()builtinfunctioncreatesafileobject.Ittakesasarguments(1)the
filenameand(2)amode.Commonlyusedmodesare"r"(read),"w"(write),and
"a"(append).
SeeBuiltinFunctions:open()
http://docs.python.org/library/functions.html#openformoreinformationon
openingfiles.SeeBuiltinTypes:FileObjects
http://docs.python.org/library/stdtypes.html#fileobjectsformoreinformationon
howtousefileobjects.
Readinganentirefileexample:
>>>infile=file('tmp.txt','r')
>>>content=infile.read()
>>>printcontent
Thisisline#1
Thisisline#2
Thisisline#3
>>>infile.close()
Notes:
Alsoconsiderusingsomethinglikecontent.splitlines(),ifyouwantto
dividecontentinlines(splitonnewlinecharacters).
Readingafileonelineatatimeexample:
>>>forlineininfile:
Page16
...print'Line:',line
...
Line:Thisisline#1
Line:Thisisline#2
Line:Thisisline#3
>>>infile.close()
Notes:
Learnmoreaboutthefor:statementinsectionfor:statement.
"infile.readlines()"returnsalistoflinesinthefile.Forlargefilesusethefile
objectitselfor"infile.xreadlines()",bothofwhichareiteratorsforthelinesinthe
file.
InolderversionsofPython,afileobjectisnotitselfaniterator.Inthoseolder
versionsofPython,youmayneedtouseinfile.readlines()orawhile
loopcontaininginfile.readline()Forexample:
>>>forlineininfile.readlines():
...print'Line:',line
...
Afewadditionalcommentsaboutstrings:
Astringisaspecialkindofsequence.So,youcanindexintothecharactersofa
stringandyoucaniterateoverthecharactersinastring.Forexample:
>>>s1='abcd'
>>>s1[1]
'b'
>>>s1[2]
'c'
>>>forchins1:
...printch
...
a
b
c
d
Ifyouneedtodofastorcomplexstringsearches,thereisaregularexpression
moduleinthestandardlibrary.rereRegularexpressionoperations
http://docs.python.org/library/re.html.
Aninterestingfeatureofstringformattingistheabilitytousedictionariesto
supplythevaluesthatareinserted.Hereisanexample:
names={'tree':'sycamore','flower':'poppy','herb':
'arugula'}
print'Thetreeis%(tree)s'%names
print'Thefloweris%(flower)s'%names
print'Theherbis%(herb)s'%names
Page17
1.4.2SequencesListsandTuples
1.4.2.1Whatsequencesare
ThereareseveraltypesofsequencesinPython.We'vealreadydiscussedstrings,which
aresequencesofcharacters.Inthissectionwewilldescribelistsandtuples.SeeBuiltin
Types:SequenceTypesstr,unicode,list,tuple,buffer,xrange
http://docs.python.org/library/stdtypes.html#sequencetypesstrunicodelisttuplebuffer
xrangeformoreinformationonPython'sbuiltinsequencetypes.
Listsaredynamicarrays.Theyarearraysinthesensethatyoucanindexitemsinalist
(forexample"mylist[3]")andyoucanselectsubranges(forexample"mylist[2:4]").
Theyaredynamicinthesensethatyoucanaddandremoveitemsafterthelistiscreated.
Tuplesarelightweightlists,butdifferfromlistsinthattheyareimmutable.Thatis,once
atuplehasbeencreated,youcannotmodifyit.Youcan,ofcourse,modifyany
(modifiable)objectsthatthetuplecontains,inotherwordsthatitrefersto.
Capabilitiesoflists:
Appendanitem.
Insertanitem(atthebeginningorintothemiddleofthelist).
Addalistofitemstoanexistinglist.
Capabilitiesoflistsandtuples:
Indexitems,thatisgetanitemoutofalistortuplebasedonthepositioninthe
list(relativetozero,thebeginningofthesequence).
Selectasubsequenceofcontiguousitems(alsoknownasaslice).
Iterateovertheitemsinthelistortuple.
1.4.2.2Whentousesequences
Wheneveryouwanttoprocessacolletionofitems.
Wheneveryouwanttoiterateoveracollectionofitems.
Wheneveryouwanttoindexintoacollectionofitems.
CollectionsNotallcollectionsinPythonareorderedsequences.Hereisacomparison
ofsomedifferenttypesofcollectionsinPythonandtheircharacteristics:
Stringordered,characters,immutable
Tupleordered,heterogeneous,immutable
Listordered,heterogeneous,mutable
Dictionaryunordered,key/valuespairs,mutable
Setunordered,heterogeneous,mutable,uniquevalues
1.4.2.3Howtousesequences
Tocreatealistusesquarebrackets.Examples:
Page18
>>>items=[111,222,333]
>>>items
[111,222,333]
Createanewlistorcopyanexistingonewiththelistconstructor:
>>>trees1=list(['oak','pine','sycamore'])
>>>trees1
['oak','pine','sycamore']
>>>trees2=list(trees1)
>>>trees2
['oak','pine','sycamore']
>>>trees1istrees2
False
Tocreateatuple,usecommas,andpossiblyparenthesesaswell:
>>>a=(11,22,33,)
>>>b='aa','bb'
>>>c=123,
>>>a
(11,22,33)
>>>b
('aa','bb')
>>>c
(123,)
>>>type(c)
<type'tuple'>
Notes:
Tocreateatuplecontainingasingleitem,westillneedthecomma.Example:
>>>print('abc',)
('abc',)
>>>type(('abc',))
<type'tuple'>
Toaddanitemtotheendofalist,useappend():
>>>items.append(444)
>>>items
[111,222,333,444]
Toinsertanitemintoalist,useinsert().Thisexampleinsertsanitematthe
beginningofalist:
>>>items.insert(0,1)
>>>items
[1,111,222,333,444]
Toaddtwoliststogether,creatinganewlist,usethe+operator.Toaddtheitemsinone
listtoanexistinglist,usetheextend()method.Examples:
>>>a=[11,22,33,]
>>>b=[44,55]
>>>c=a+b
Page19
>>>c
[11,22,33,44,55]
>>>a
[11,22,33]
>>>b
[44,55]
>>>a.extend(b)
>>>a
[11,22,33,44,55]
Youcanalsopushitemsontotherightendofalistandpopitemsofftherightendofa
listwithappend()andpop().Thisenablesustousealistasastacklikedata
structure.Example:
>>>items=[111,222,333,444,]
>>>items
[111,222,333,444]
>>>items.append(555)
>>>items
[111,222,333,444,555]
>>>items.pop()
555
>>>items
[111,222,333,444]
And,youcaniterateovertheitemsinalistortuple(orothercollection,forthatmatter)
withthefor:statement:
>>>foriteminitems:
...print'item:',item
...
item:1
item:111
item:222
item:333
item:444
Formoreonthefor:statement,seesectionfor:statement.
1.4.3Dictionaries
1.4.3.1Whatdictionariesare
Adictionaryis:
Anassociativearray.
Amappingfromkeystovalues.
Acontainer(collection)thatholdskeyvaluepairs.
Adictionaryhasthefollowingcapabilities:
Abilitytoiterateoverkeysorvaluesorkeyvaluepairs.
Abilitytoaddkeyvaluepairsdynamically.
Page20
Abilitytolookupavaluebykey.
Forhelpondictionaries,type:
>>>helpdict
atPython'sinteractiveprompt,or:
$pydocdict
atthecommandline.
Italsomaybehelpfultousethebuiltindir()function,thentoaskforhelpona
specificmethod.Example:
>>>a={}
>>>dir(a)
['__class__','__cmp__','__contains__','__delattr__',
'__delitem__','__doc__','__eq__','__format__','__ge__',
'__getattribute__','__getitem__','__gt__','__hash__','__init__',
'__iter__','__le__','__len__','__lt__','__ne__','__new__',
'__reduce__','__reduce_ex__','__repr__','__setattr__',
'__setitem__','__sizeof__','__str__','__subclasshook__','clear',
'copy','fromkeys','get','has_key','items','iteritems',
'iterkeys','itervalues','keys','pop','popitem','setdefault',
'update','values']
>>>
>>>help(a.keys)
Helponbuiltinfunctionkeys:
keys(...)
D.keys()>listofD'skeys
Moreinformationaboutdictionaryobjectsisavailablehere:Mappingtypesdict
http://docs.python.org/library/stdtypes.html#mappingtypesdict.
1.4.3.2Whentousedictionaries
Whenyouneedlookupbykey.
Whenyouneeda"structured"liteweightobjectoranobjectwithnamedfields.
(But,don'tforgetclasses,whichyouwilllearnaboutlaterinthisdocument.)
Whenyouneedtomapanameorlabeltoanykindofobject,evenanexecutable
onesuchasafunction.
1.4.3.3Howtousedictionaries
Createadictionarywithcurlybracets.Itemsinadictionaryareseparatebycommas.Use
acolonbetweeneachkeyanditsassociatedvalue:
>>>lookup={}
>>>lookup
{}
>>>states={'az':'Arizona','ca':'California'}
>>>states['ca']
Page21
'California'
or:
>>>deffruitfunc():
...print"I'mafruit."
>>>defvegetablefunc():
...print"I'mavegetable."
>>>
>>>lookup={'fruit':fruitfunc,'vegetable':vegetablefunc}
>>>lookup
{'vegetable':<functionvegetablefuncat0x4028980c>,
'fruit':<functionfruitfuncat0x4028e614>}
>>>lookup['fruit']()
I'mafruit.
>>>lookup['vegetable']()
I'mavegetable.
or:
>>>lookup=dict((('aa',11),('bb',22),('cc',33)))
>>>lookup
{'aa':11,'cc':33,'bb':22}
Notethatthekeysinadictionarymustbeimmutable.Therefore,youcanuseanyofthe
followingaskeys:numbers,strings,tuples.
Testfortheexistenceofakeyinadictionarywiththeinoperator:
>>>if'fruit'inlookup:
...print'containskey"fruit"'
...
containskey"fruit"
or,alternatively,usethe(slightlyoutdated)has_key()method:
>>>iflookup.has_key('fruit'):
...print'containskey"fruit"'
...
containskey"fruit"
Accessthevalueassociatedwithakeyinadictionarywiththeindexingoperator(square
brackets):
>>>printlookup['fruit']
<functionfruitfuncat0x4028e614>
Noticethattheabovewillthrowanexceptionifthekeyisnotinthedictionary:
>>>printlookup['salad']
Traceback(mostrecentcalllast):
File"<stdin>",line1,in<module>
KeyError:'salad'
Andso,theget()methodisaneasywaytogetavaluefromadictionarywhile
avoidinganexception.Forexample:
Page22
>>>printlookup.get('fruit')
>>>printlookup.get('salad')
None
>>>printlookup.get('salad',fruitfunc)
Adictionaryisaniteratorobjectthatproducesitskeys.So,wecaniterateoverthekeys
inadictionaryasfollows:
>>>forkeyinlookup:
...print'key:%s'%key
...lookup[key]()
...
key:vegetable
I'mavegetable.
key:fruit
I'mafruit.
And,rememberthatyoucansubclassdictionaries.Herearetwoversionsofthesame
example.ThekeywordargumentsinthesecondversionrequirePython2.3orlater:
#
#ThisexampleworkswithPython2.2.
classMyDict_for_python_22(dict):
def__init__(self,**kw):
forkeyinkw.keys():
self[key]=kw[key]
defshow(self):
print'ShowingexampleforPython2.2...'
forkeyinself.keys():
print'key:%svalue:%s'%(key,self[key])
deftest_for_python_22():
d=MyDict_for_python_22(one=11,two=22,three=33)
d.show()
test_for_python_22()
AversionfornewerversionsofPython:
#
#ThisexampleworkswithPython2.3ornewerversionsofPython.
#Keywordsupport,whensubclassingdictionaries,seemstohave
#beenenhancedinPython2.3.
classMyDict(dict):
defshow(self):
print'ShowingexampleforPython2.3ornewer.'
forkeyinself.keys():
print'key:%svalue:%s'%(key,self[key])
deftest():
d=MyDict(one=11,two=22,three=33)
d.show()
Page23
test()
Runningthisexampleproduces:
ShowingexampleforPython2.2...
key:onevalue:11
key:threevalue:33
key:twovalue:22
ShowingexampleforPython2.3ornewer.
key:threevalue:33
key:twovalue:22
key:onevalue:11
Afewcommentsaboutthisexample:
LearnmoreaboutclassesandhowtoimplementtheminsectionClassesand
instances.
TheclassMyDictdoesnotdefineaconstructor(__init__).Thisenablesusto
reusethecontructorfromsuperclassdictandanyofitsforms.Type"help
dict"atthePythoninteractiveprompttolearnaboutthevariouswaystocallthe
dictconstructor.
Theshowmethodisthespecializationaddedtooursubclass.
Inoursubclass,wecanrefertoanymethodsinthesuperclass(dict).For
example:self.keys().
Inoursubclass,wecanreferthedictionaryitself.Forexample:self[key].
1.4.4Files
1.4.4.1Whatfilesare
AfileisaPythonobjectthatgivesusaccesstoafileonthedisksystem.
Afileobjectcanbecreated("opened")forreading("r"mode),forwriting("w"
mode),orforappending("a"mode)toafile.
Openingafileforwritingerasesanexistingwiththatpath/name.Openingafile
forappenddoesnot.
1.4.4.2Whentousefiles
Useafileobjectanytimeyouwishtoreadfromorwritetothediskfilesystem.
1.4.4.3Howtousefiles
Hereisanexamplethat(1)writestoafile,then(2)appendstothatfile,andfinally,(3)
readsfromthefile:
defwrite_file(outfilename):
outfile=open(outfilename,'w')
outfile.write('Line#1\n')
Page24
outfile.close()
defappend_file(outfilename):
outfile=open(outfilename,'a')
outfile.close()
defread_file(infilename):
infile=open(infilename,'r')
forlineininfile:
printline.rstrip()
infile.close()
deftest():
filename='temp_file.txt'
write_file(filename)
read_file(filename)
append_file(filename)
print''*50
read_file(filename)
test()
1.4.4.4ReadingTextFiles
Toreadatextfile,firstcreateafileobject.Hereisanexample:
inFile=open('messages.log','r')
Thenusethefileobjectasaniteratororuseoneormoreofthefileobject'smethodsto
processthecontentsofthefile.Hereareafewstrategies:
UseforlineininFile:toprocessonelineatatime.Youcandothis
because(atleastsincePython2.3)fileobjectsobeytheiteratorprotocol,thatis
theysupportmethods__iter__()andnext().Formoreontheiterator
protocolseePythonStandardLibrary:IteratorTypes
http://docs.python.org/library/stdtypes.html#iteratortypes.
Example:
>>>inFile=file('tmp.txt','r')
>>>forlineininFile:
...print'Line:',line,
...
Line:aaaaa
Line:bbbbb
Line:ccccc
Line:ddddd
Line:eeeee
>>>inFile.close()
ForearlierversionsofPython,onestrategyistouse"inFile.readlines()",which
createsalistoflines.
Page25
Ifyourwanttogetthecontentsofanentiretextfileasacollectionoflines,use
readlines().Alternatively,youcoulduseread()followedby
splitlines().Example:
>>>inFile=open('data2.txt','r')
>>>lines=inFile.readlines()
>>>inFile.close()
>>>lines
['aaabbbccc\n','dddeeefff\n','ggghhhiii\n']
>>>
>>>inFile=open('data2.txt','r')
>>>content=inFile.read()
>>>inFile.close()
>>>lines=content.splitlines()
>>>lines
['aaabbbccc','dddeeefff','ggghhhiii']
UseinFile.read()togettheentirecontentsofthefile(astring).Example:
>>>inFile=open('tmp.txt','r')
>>>content=inFile.read()
>>>inFile.close()
>>>printcontent
aaabbbccc
dddeeefff
ggghhhiii
>>>words=content.split()
>>>printwords
['aaa','bbb','ccc','ddd','eee','fff','ggg',
'hhh','iii']
>>>forwordinwords:
...printword
...
aaa
bbb
ccc
ddd
eee
fff
ggg
hhh
iii
1.5SimpleStatements
SimplestatementsinPythondonotcontainanestedblock.
1.5.1printstatement
Alert:InPythonversion3.0,theprintstatementhasbecometheprint()builtin
function.Youwillneedtoaddparentheses.
Theprintstatementsendsoutputtostdout.
Hereareafewexamples:
Page26
printobj
printobj1,obj2,obj3
print"Mynameis%s"%name
Notes:
Toprintmultipleitems,separatethemwithcommas.Theprintstatementinsertsa
blankbetweenobjects.
Theprintstatementautomaticallyappendsanewlinetooutput.Toprintwithouta
newline,addacommaafterthelastobject,oruse"sys.stdout",forexample:
print'Outputwithnonewline',
whichwillappendablank,or:
importsys
sys.stdout.write("Someoutput")
Toredefinethedestinationofoutputfromtheprintstatement,replacesys.stdout
withaninstanceofaclassthatsupportsthewritemethod.Forexample:
importsys
classWriter:
def__init__(self,filename):
self.filename=filename
defwrite(self,msg):
f=file(self.filename,'a')
f.write(msg)
f.close()
sys.stdout=Writer('tmp.log')
print'Logmessage#1'
print'Logmessage#2'
print'Logmessage#3'
MoreinformationontheprintstatementisatTheprintstatement
http://docs.python.org/reference/simple_stmts.html#theprintstatement.
1.5.2Assignmentstatement
Theassignmentoperatoris=.
Herearesomeofthethingsyoucanassignavalueto:
Aname(variable)
Anitem(position)inalist.Example:
>>>a=[11,22,33]
>>>a
[11,22,33]
>>>a[1]=99
>>>a
[11,99,33]
Akeyinadictionary.Example:
>>>names={}
>>>names['albert']=25
>>>names
Page27
{'albert':25}
Asliceinalist.Example:
>>>a=[11,22,33,44,55,66,77,]
>>>a
[11,22,33,44,55,66,77]
>>>a[1:3]=[999,888,777,666]
>>>a
[11,999,888,777,666,44,55,66,77]
Atupleorlist.Assignmenttoatupleorlistperformsunpacking.Example:
>>>values=111,222,333
>>>values
(111,222,333)
>>>a,b,c=values
>>>a
111
>>>b
222
>>>c
333
Unpackingsuggestsaconvenientidiomforreturningandcapturingamultiple
argumentsfromafunction.Example:
>>>defmultiplier(n):
...returnn,n*2,n*3
...
>>>
>>>x,y,z=multiplier(4)
>>>x
4
>>>y
8
>>>z
12
Ifafunctionneedstoreturnavariablenumberofvalues,thenunpackingwillnot
do.But,youcanstillreturnmultiplevaluesbyreturningacontainerofsomekind
(forexample,atuple,alist,adictionary,aset,etc.).
Anattribute.Example:
>>>classA(object):
...pass
...
>>>c=A()
>>>
>>>a=A()
>>>a.size=33
>>>printa.size
33
>>>a.__dict__
{'size':33}
1.5.3importstatement
Thingstoknowabouttheimportstatement:
Page28
Theimportstatementmakesamoduleanditscontentsavailableforuse.
Theimportstatementevaluatesthecodeinamodule,butonlythefirsttimethat
anygivenmoduleisimportedinanapplication.
Allmodulesinanapplicationthatimportagivenmoduleshareasinglecopyof
thatmodule.Example:ifmodulesAandBbothimportmoduleC,thenAandB
shareasinglecopyofC.
Hereareseveralformsoftheimportstatement:
Importamodule.Refertoanattributeinthatmodule:
importtest
printtest.x
Importaspecificattributefromamodule:
fromtestimportx
fromothertestimporty,z
printx,y,z
Importalltheattributesinamodule:
fromtestimport*
printx
printy
Recommendation:Usethisformsparingly.Usingfrommodimport*
makesitdifficulttotrackdownvariablesand,thus,todebugyourcode
Importamoduleandrenameit.Importanattributefromamoduleandrenameit:
importtestastheTest
fromtestimportxastheValue
printtheTest.x
printtheValue
Afewcommentsaboutimport:
Theimportstatementalsoevaluatesthecodeintheimportedmodule.
But,thecodeinamoduleisonlyevaluatedthefirsttimeitisimportedina
program.So,forexample,ifamodulemymodule.pyisimportedfromtwoother
modulesinaprogram,thestatementsinmymodulewillbeevaluatedonlythefirst
timeitisimported.
Ifyouneedevenmorevarietythattheimportstatementoffers,seetheimp
module.DocumentationisatimpAccesstheimportinternals
http://docs.python.org/library/imp.html#moduleimp.Alsoseethe
__import__()builtinfunction,whichyoucanreadabouthere:Builtin
Functions:__import()
http://docs.python.org/library/functions.html#__import__.
MoreinformationonimportisatLanguageReference:Theimportstatement
http://docs.python.org/reference/simple_stmts.html#theimportstatement.
1.5.4assertstatement
Usetheassertstatementtoplaceerrorcheckingstatementsinyourcode.Hereisan
example:
Page29
deftest(arg1,arg2):
arg1=float(arg1)
arg2=float(arg2)
assertarg2!=0,'Baddividendarg1:%farg2:%f'%(arg1,
arg2)
ratio=arg1/arg2
print'ratio:',ratio
Whenarg2iszero,runningthiscodewillproducesomethinglikethefollowing:
File"tmp.py",line22,in?
main()
File"tmp.py",line18,inmain
test(args[0],args[1])
File"tmp.py",line8,intest
assertarg2!=0,'Baddividendarg1:%farg2:%f'%(arg1,
arg2)
AssertionError:Baddividendarg1:2.000000arg2:0.000000
Afewcomments:
Noticethatthetracebackidentifiesthefileandlinewherethetestismadeand
showsthetestitself.
Ifyourunpythonwiththeoptimizeoptions(OandOO),theassertiontestisnot
performed.
Thesecondargumenttoassert()isoptional.
1.5.5globalstatement
TheproblemImagineaglobalvariableNAME.If,inafunction,thefirstmentionof
thatvariableis"name=NAME",thenI'llgetthevalueofthetheglobalvariableNAME.
But,if,inafunction,myfirstmentionofthatvariableisanassignmenttothatvariable,
thenIwillcreateanewlocalvariable,andwillnotrefertotheglobalvariableatall.
Consider:
NAME="Peach"
defshow_global():
name=NAME
print'(show_global)name:%s'%name
defset_global():
NAME='Nectarine'
name=NAME
print'(set_global)name:%s'%name
show_global()
set_global()
show_global()
Runningthiscodeproduces:
Page30
(show_global)name:Peach
(set_global)name:Nectarine
Theset_globalmodifiesalocalvariableandnottheglobalvariableasImighthave
intended.
ThesolutionHowcanIfixthat?Hereishow:
NAME="Peach"
defshow_global():
name=NAME
print'(show_global)name:%s'%name
defset_global():
globalNAME
NAME='Nectarine'
name=NAME
print'(set_global)name:%s'%name
show_global()
set_global()
show_global()
Noticetheglobalstatementinfunctionset_global.Runningthiscodedoesmodifythe
globalvariableNAME,andproducesthefollowingoutput:
(set_global)name:Nectarine
(show_global)name:Nectarine
Comments:
Youcanlistmorethanoneveriableintheglobalstatement.Forexample:
globalNAME1,NAME2,NAME3
1.6CompoundstatmentsControlStructures
Acompoundstatementhasanested(andindented)blockofcode.Acompoundstatement
mayhavemultipleclauses,eachwithanestedblockofcode.Eachcompoundstatement
hasaheaderline(whichstartswithakeywordandendswithacolon).
1.6.1if:statement
Theifstatementenablesustoexecutecode(ornot)dependingonacondition:
ifcondition:
statementblock
ifcondition:
statementblock1
else:
statementblock2
Page31
ifcondition1:
statementblock1
elifcondition2:
statementblock2
o
o
o
else:
statementblockn
Hereisanexample:
>>>y=25
>>>
>>>ify>15:
...print'yislarge'
...else:
...print'yissmall'
...
yislarge
Afewnotes:
Theconditioncanbeanyexpression,i.e.somethingthatreturnsavalue.A
detaileddescriptionofexpressionscanbefoundatPythonLanguageReference:
Expressionshttp://docs.python.org/reference/expressions.html.
Parenthesesarenotneededaroundthecondition.Useparenthesestogroupsub
expressionsandcontroltheorderofevaluationwhenthenaturaloperator
precedenceisnotwhatyouwant.Python'soperatorprecedencesaredescribedat
PythonLanguageReference:Expressions:Summary
http://docs.python.org/reference/expressions.html#summary.
Pythonhasnoswitchstatement.Useif:elif:....Orconsiderusinga
dictionary.Hereisanexamplethatusesbothofthesetechniques:
deffunction1():
print"Hi.I'mfunction1."
deffunction2():
deffunction3():
deferror_function():
print"Invalidoption."
deftest1():
while1:
code=raw_input('Enter"one","two","three",
or"quit":')
ifcode=='quit':
break
ifcode=='one':
function1()
elifcode=='two':
function2()
elifcode=='three':
Page32
function3()
else:
error_function()
deftest2():
mapper={'one':function1,'two':function2,
'three':function3}
while1:
code=raw_input('Enter"one","two","three",
or"quit":')
ifcode=='quit':
break
func=mapper.get(code,error_function)
func()
deftest():
test1()
print''*50
test2()
if__name__=='__main__':
test()
1.6.2for:statement
Thefor:statementenablesustoiterateovercollections.Itenablesustorepeatasetof
linesofcodeonceforeachiteminacollection.Collectionsarethingslikestrings(arrays
ofcharacters),lists,tuples,anddictionaries.
Hereisanexample:
>>>collection=[111,222,333]
>>>foritemincollection:
...print'item:',item
...
item:111
item:222
item:333
Comments:
Youcaniterateoverstrings,lists,andtuples.Actually,youcaniterateover
almostanycontainerlikeobject.
Iterateoverthekeysorvaluesinadictionarywith"aDict.keys()"and
"aDict.values()".Hereisanexample:
>>>aDict={'cat':'furryandcute','dog':'friendly
andsmart'}
>>>aDict.keys()
['dog','cat']
>>>aDict.values()
['friendlyandsmart','furryandcute']
>>>forkeyinaDict.keys():
...print'A%sis%s.'%(key,aDict[key])
...
Page33
Adogisfriendlyandsmart.
Acatisfurryandcute.
InrecentversionsofPython,adictionaryitselfisaniteratorforitskeys.
Therefore,youcanalsodothefollowing:
>>>forkeyinaDict:
...print'A%sis%s.'%(key,aDict[key])
...
Adogisfriendlyandsmart.
Acatisfurryandcute.
And,inrecentversionsofPython,afileisalsoaniteratoroverthelinesinthe
file.Therefore,youcandothefollowing:
>>>forlineininfile:
...printline,
...
Thisisline#1
Thisisline#2
Thisisline#3
>>>infile.close()
Thereareotherkindsofiterators.Forexample,thebuiltiniterwillproducean
iteratorfromacollection.Hereisanexample:
deftest():
anIter=iter([11,22,33])
foriteminanIter:
print'item:',item
test()
Whichproduces:
item:11
item:22
item:33
Youcanalsoimplementiteratorsofyourown.Onewaytodoso,istodefinea
functionthatreturnsvalueswithyield(insteadofwithreturn).Hereisan
example:
deft(collection):
icollection=iter(collection)
foriteminicollection:
yield'||%s||'%item
deftest():
collection=[111,222,333,]
forxint(collection):
printx
test()
Whichprintsout:
||111||
||222||
||333||
Page34
1.6.2.1Thefor:statementandunpacking
Ifaniteratorproducesasequenceoflistsortuples,eachofwhichcontainthesame
(small)numberofitems,thenyoucandounpackingdirectlyintheheaderofthefor:
statement.Hereisanexample:
In[5]:collection=[('apple','red'),('banana','yello'),('kiwi',
'green')]
In[6]:forname,colorincollection:
...:print'name:%s,color:%s'%(name,color,)
...:
...:
name:apple,color:red
name:banana,color:yello
name:kiwi,color:green
Theunpackingdescribedaboveandtheenumeratebuiltinfunctionprovidesa
convenientwaytoprocessitemsinasequencewithanindex.Thisexampleaddseach
valueinonelisttothecorrespondingvalueinasecondlist:
In[9]:a=[11,22,33]
In[10]:a=[11,22,33]
In[11]:
In[12]:a=[11,22,33]
In[13]:b=[111,222,333]
In[14]:foridx,valueinenumerate(a):
....:b[idx]+=value
....:
....:
In[15]:a
Out[15]:[11,22,33]
In[16]:b
Out[16]:[122,244,366]
Anotherwaytoimplementaniteratoristoimplementaclassthatsupportstheiterator
protocol.SeePythonStandardLibrary:IteratorTypes
http://docs.python.org/library/stdtypes.html#iteratortypesformoreonimplementing
iterators.ButnoticethatthisprotocolhaschangedinPython3.0.Forinformationonthe
iteratorprotocolinPython3.0,seePython3.0StandardLibrary:IteratorTypes
http://docs.python.org/3.0/library/stdtypes.html#iteratortypes.
1.6.3while:statement
while:isanotherrepeatingstatement.Itexecutesablockofcodeuntilaconditionis
false.
Hereisanexample:
>>>reply='repeat'
>>>whilereply=='repeat':
...print'Hello'
Page35
...reply=raw_input('Enter"repeat"todoitagain:')
...
Hello
Enter"repeat"todoitagain:repeat
Hello
Enter"repeat"todoitagain:bye
Comments:
Usethebreakstatementtoexitimmediatelyfromaloop.Thisworksinboth
for:andwhile:.Hereisanexamplethatusesbreakinafor:statement:
#for_break.py
"""Countlinesuntilalinethatbeginswithadouble
#.
"""
importsys
defcountLines(infilename):
infile=file(infilename,'r')
count=0
forlineininfile.readlines():
line=line.strip()
ifline[:2]=='##':
break
count+=1
returncount
defusage():
print'Usage:pythonpython_101_for_break.py
<infilename>'
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)!=1:
usage()
count=countLines(args[0])
print'count:',count
main()
Usethecontinuestatementtoskiptheremainderofthecodeblockinafor:
orwhile:statement.Acontinueisashortcircuitwhich,ineffect,branches
immediatelybacktothetopofthefor:orwhile:statement(orifyouprefer,
totheendoftheblock).
Thetestif__name__=='__main__':isusedtoenableascripttoboth
be(1)importedand(2)runfromthecommandline.Thatconditionistrueonly
whenthescriptisrun,butnotimported.ThisisacommonPythonidiom,which
youshouldconsiderincludingattheendofyourscripts,whether(1)togiveyour
usersademonstrationofwhatyourscriptdoesandhowtouseitor(2)toprovide
atestofthescript.
Page36
1.6.4try:except:andraiseExceptions
Useatry:except:statementtocatchanexception.
Usetheraisestatementtoraiseanexception.
Commentsandhints:
Catchallexceptionswitha"bare"except:.Forexample:
>>>try:
...x=y
...except:
...print'ynotdefined'
...
ynotdefined
Note,however,thatitisusuallybettertocatchspecificexceptions.
Catchaspecificerrorbyreferingtoanexceptionclassintheexcept:.To
determinewhaterrororexceptionyouwanttocatch,generateitandtryit.
BecausePythonreportserrorswithawalkbackthatendswithreportingthe
exception,youcanlearnwhichexceptiontocatch.Forexample,supposeIwant
tolearnwhichexceptionisthrownwhenaPythoncan'topenafile.Icantrythe
followingfromtheinteractiveprompt:
>>>myfile=file('amissingfile.py','r')
File"<stdin>",line1,in?
IOError:[Errno2]Nosuchfileordirectory:
'amissingfile.py'
So,nowIknowthatIcando:
deftest():
try:
myfile=file('amissingfile.py','r')
exceptIOError:
print'amissingfile.pyismissing'
test()
Whichproduces:
amissingfile.pyismissing
Exceptiontypesaredescribedhere:PythonStandardLibrary:BuiltinExceptions
http://docs.python.org/library/exceptions.html.
Catchanyoneofseveralexecptiontypesbyusingatuplecontainingthe
exceptionstobecaught.Example:
try:
f=open('abcdexyz.txt','r')
d={}
x=d['name']
except(IOError,KeyError),e:
print'Theerroris',e
Notethatmultipletypesofexceptionstobecaughtbyasingleexcept:clauseare
inparentheses;theyareatuple.
Page37
Youcancustomizeyourerrorhandlingstillfurther(1)bypassinganobjectwhen
youraisetheexceptionand(2)bycatchingthatobjectintheexcept:clauseof
yourtry:statement.Bydoingso,youcanpassinformationupfromtheraise
statementtoanexceptionhandler.Onewayofdoingthisistopassanobject.A
reasonablestrategyistodefineasubclassofastandardexception.Forexample:
classE(Exception):
def__init__(self,msg):
self.msg=msg
defgetMsg(self):
returnself.msg
deftest():
try:
raiseE('mytesterror')
exceptE,obj:
print'Msg:',obj.getMsg()
test()
Whichproduces:
Msg:mytesterror
Ifyoucatchanexceptionusingtry:except:,butthenfindthatyoudonot
wanttohandletheexceptionatthatlocation,youcan"reraise"thesame
exception(withthesamearguments)byusingraisewithnoarguments.An
example:
classGeneralException(Exception):
pass
classSimpleException(GeneralException):
pass
classComplexException(GeneralException):
pass
defsome_func_that_throws_exceptions():
#raiseSimpleException('thisisasimpleerror')
raiseComplexException('thisisacomplexerror')
deftest():
try:
some_func_that_throws_exceptions()
exceptGeneralException,e:
ifisinstance(e,SimpleException):
printe
else:
raise
test()
1.7Organization
ThissectiondescribesPythonfeaturesthatyoucanusetoorganizeandstructureyour
code.
Page38
1.7.1Functions
1.7.1.1Abasicfunction
Usedeftodefineafunction.Hereisasimpleexample:
deftest(msg,count):
foridxinrange(count):
print'%s%d'%(msg,idx)
test('Test#',4)
Comments:
Afterevaluationdefcreatesafunctionobject.
Callthefunctionusingtheparenthesesfunctioncallnotation,inthiscase
"test('Test#',4)".
AswithotherPythonobjects,youcanstuffafunctionobjectintootherstructures
suchastuples,lists,anddictionaries.Hereisanexample:
#Createatuple:
val=(test,'Alabel:',5)
#Callthefunction:
val[0](val[1],val[2])
1.7.1.2Afunctionwithdefaultarguments
Providingdefaultargumentsallowsthecallertoomitsomearguments.Hereisan
example:
deftestDefaultArgs(arg1='default1',arg2='default2'):
print'arg1:',arg1
print'arg2:',arg2
testDefaultArgs('Explicitvalue')
Theaboveexampleprints:
arg1:Explicitvalue
arg2:default2
1.7.1.3Argumentlistsandkeywordargumentlists
Hereisanexample:
deftestArgLists_1(*args,**kwargs):
print'args:',args
print'kwargs:',kwargs
testArgLists_1('aaa','bbb',arg1='ccc',arg2='ddd')
Page39
deftestArgLists_2(arg0,*args,**kwargs):
print'arg0:"%s"'%arg0
print'args:',args
deftest():
testArgLists_1('aaa','bbb',arg1='ccc',arg2='ddd')
print'='*40
testArgLists_2('afirstargument','aaa','bbb',arg1='ccc',
arg2='ddd')
test()
Runningthisexampledisplays:
args:('aaa','bbb')
kwargs:{'arg1':'ccc','arg2':'ddd'}
========================================
arg0:"afirstargument"
args:('aaa','bbb')
Alittleguidance:
Positionalargumentsmustproceedallkeywordargumentswhenyoucallthe
function.
Youcanalsohave"normal"argumentsinthefunctiondefinition.Forexample:
deftest(arg0,*args,**kwargs):.Seethesecondexampleabove.
Thevalueofthekeywordparameter(**kwargs)isadictionary,soyoucando
anythingwithitthatyoudowithanormaldictionary.
1.7.1.4Callingafunctionwithkeywordarguments
Youcanalsocallafunctionusingthenameofaparameterasakeyword.Hereisan
example:
deftest_keyword_args(foreground_color='black',
background_color='white',
link_color='blue',
visited_link_color='red'):
print'foreground_color:"%s"'%foreground_color
print'background_color:"%s"'%background_color
print'link_color:"%s"'%link_color
print'visited_link_color:"%s"'%visited_link_color
deftest():
test_keyword_args()
print''*40
test_keyword_args(background_color='green')
print''*40
test_keyword_args(link_color='gray',
visited_link_color='yellow')
test()
Page40
Whenwerunthisexample,itproducesthefollowing:
foreground_color:"black"
background_color:"white"
link_color:"blue"
visited_link_color:"red"
background_color:"green"
link_color:"blue"
visited_link_color:"red"
background_color:"white"
link_color:"gray"
visited_link_color:"yellow"
1.7.2Classesandinstances
1.7.2.1Abasicclass
Defineabasicclassasfollows:
classBasic:
def__init__(self,name):
self.name=name
defshow(self):
print'Basicname:%s'%self.name
deftest():
obj1=Basic('Apricot')
obj1.show()
test()
Runningtheaboveexampleproducesthefollowing:
Basicname:Apricot
Explanation:
Methodsareaddedtotheclasswithdef.Thefirstargumenttoamethodisthe
classinstance.Byconventionitisspelled"self".
Theconstructorforaclassisamethodnamed__init__.
Theselfvariablemustbeexplicitlylistedasthefirstargumenttoamethod.You
couldspellitdifferentlyfrom"self",butdon'tdoso.
Instancevariablesarereferredtowith"self.XXX".Noticehowinourexamplean
argumenttotheconstructorissavedasaninstancevariable.
Aninstanceiscreatedby"calling"theclass.Forexample:obj=
Basic('Apricot').
Inadditionto__init__thereareotherspecialmethodnamesoftheform
Page41
"__XXX__",whichareusedtocustomizeclassesandtheirinstances.Theseare
describedatPythonLanguageReference:Specialmethodnames
http://docs.python.org/reference/datamodel.html#specialmethodnames
<http://docs.python.org/reference/datamodel.html#specialmethodnames>_.
Afewmorenotesonself:
selfisareferencetotheinstance.Thinkofit(inpart)asareferencetothe
containerforthedataorstatefortheobject.
Inmanyobjectorientedprogramminglanguages,theinstanceishiddeninthe
methoddefinitions.Theselanguagestypicallyexplainthisbysayingsomething
like"Theinstanceispassedasanimplicitfirstargumenttothemethod."
InPython,theinstanceisvisibleandexplicitinmethoddefinitions.Youmust
explicitlydeclaretheinstanceasthefirstparameterofeach(instance)method.
Thisfirstparameteris(almost)alwaysspelled"self".
1.7.2.2Inheritance
DefineaclassSpecialthatinheritsfromasuperclassBasicasfollows:
classBasic:
self.name=name
defshow(self):
print'Basicname:%s'%self.name
classSpecial(Basic):
def__init__(self,name,edible):
Basic.__init__(self,name)
self.upper=name.upper()
self.edible=edible
defshow(self):
Basic.show(self)
print'Specialuppername:%s.'%self.upper
ifself.edible:
print"It'sedible."
else:
print"It'snotedible."
defedible(self):
returnself.edible
deftest():
obj1=Basic('Apricot')
obj1.show()
print'='*30
obj2=Special('Peach',1)
obj2.show()
test()
Runningthisexampleproducesthefollowing:
Basicname:Apricot
Page42
==============================
Basicname:Peach
Specialuppername:PEACH.It'sedible.
Comments:
Thesuperclassislistedaftertheclassnameinparentheses.Formultiple
inheritence,separatethesuperclasseswithcommas.
Callamethodinthesuperclass,bypassingthemethodwiththesamenamein
thesubclass,fromthesubclassbyusingthesuperclassname.Forexample:
Basic.__init__(self,name)andBasic.show(self).
Inourexample(above),thesubclass(Special)specializesthesuperclass
(Basic)byaddingadditionalmembervariables(self.upperand
self.edible)andbyaddinganadditionalmethod(edible).
1.7.2.3Classdata
Aclassdatamemberisamemberthathasonlyonevaluefortheclassandallits
instances.HereisanexamplefromthePythonFAQat
http://www.python.org/doc/FAQ.html:
classC:
count=0#numberoftimesC.__init__called
def__init__(self):
C.count+=1
defgetcount(self):
returnC.count#orreturnself.count
deftest():
c1=C()
print'Currentcount:',c1.getcount()
c2=C()
print'Currentcount:',c2.getcount()
test()
Runningthisexampleproduces:
Currentcount:1
Currentcount:2
1.7.2.4Staticmethodsandclassmethods
Newstyleclassescanhavestaticmethodsandclassmethods.
Anewstyleclassisaclassthatinheritsdirectlyorindirectlyfromobjectorfromabuilt
intype.
Hereisanexamplethatshowshowtodefinestaticmethodsandclassmethods:
classAdvanced(object):
Page43
self.name=name
defDescription():
return'Thisisanadvancedclass.'
defClassDescription(cls):
return'Thisisadvancedclass:%s'%repr(cls)
Description=staticmethod(Description)
ClassDescription=classmethod(ClassDescription)
obj1=Advanced('Nectarine')
printobj1.Description()
printobj1.ClassDescription()
print'='*30
printAdvanced.Description()
printAdvanced.ClassDescription()
Runningtheaboveproducesthefollowingoutput:
Thisisanadvancedclass.
Thisisadvancedclass:<class__main__.Advancedat0x401c926c>
==============================
Thisisanadvancedclass.
Thisisadvancedclass:<class__main__.Advancedat0x401c926c>
Notes:
Theclassinheritsfromclassobject,whichmakesitanewstyleclass.
Createastaticmethodwithx=staticmethod(y),whereyisanormal
methodbutwithouttheself/firstparameter.
Createaclassmethodwithx=classmethod(y),whereyisanormal
method.
Thedifferencebetweenstaticandclassmethodsisthataclassmethodreceivestheclass
(nottheinstance)asitsfirstargument.Asummary:
Anormal/standardmethodalwaysreceivesaninstanceasitsfirstargument.
Aclassmethodalwaysreceivestheclassasitsfirstargument.
Astaticmethoddoesnot(automatically)receiveeithertheinstanceortheclassas
thefirstargument.
Youcancallstaticandclassmethodsusingeitheraninstanceoraclass.Inour
exampleeither"obj1.Description()"or"Advanced.Description()"willwork.
YoushouldalsoreviewtherelevantstandardPythondocumentationonthe
classmethodandstaticmethodbuiltinfunctions,whichyoucanfindatPython
LibraryReference2.1BuiltinFunctions
http://docs.python.org/library/functions.html.
Bynow,youarelikelytobeasking:"WhyandwhenshouldIuseclassmethodsand
staticmethods?"Hereisabitofguidance:
Mostofthetime,almostalways,implementplaininstancemethods.Implement
aninstancemethodwheneverthemethodneedsaccesstothevaluesthatare
specifictotheinstanceorneedstocallothermethodsthathaveaccesstoinstance
specificvalues.Ifthemethodneedsself,thenyouprobablyneedaninstance
Page44
method.
Implementaclassmethod(1)whenthemethoddoesnotneedaccesstoinstance
variablesand(2)whenyoudonotwanttorequirethecallerofthemethodto
createaninstanceand(3)whenthemethodneedsaccesstoclassvariables.A
classmethodmaybecalledoneitheraninstanceortheclass.Aclassmethodgets
theclassasafirstargument,whetheritiscalledontheclassortheinstance.Ifthe
methodneedsaccesstotheclassbutdoesnotneedself,thenthinkclassmethod.
Implementastaticmethodifyoumerelywanttoputthecodeofthemethod
withinthescopeoftheclass,perhapsforpurposesoforganizingyourcode,but
themethodneedsaccesstoneitherclassnorinstancevariables(thoughyoucan
accessclassvariablesthroughtheclassitself).Astaticmethodmaybecalledon
eitheraninstanceortheclass.Astaticmethodgetsneithertheclassnorthe
instanceasanargument.
Tosummarize:
Implementaninstancemethod,unless...themethodneedsaccesstoclassvariablesbut
notinstancevariables,thenimplementaclassmethod,unless...themethodneedsaccess
toneitherinstancevariablesnorclassvariablesandyoustillwanttoincludeitwithinthe
classdefinition,thenimplementastaticmethod.
Aboveall,writeclear,plaincodethatwillbeunderstandabletoyourreaders.Donotuse
amoreconfusinglanguagefeatureanddonotforceyourreaderstolearnanewlanguage
featureunlesyouhaveagoodreason.
1.7.2.5Properties
Anewstyleclasscanhaveproperties.Apropertyisanattributeofaclassthatis
associatedwithagetterandasetterfunction.
Declarethepropertyanditsgetterandsetterfunctionswithproperty().
Hereisanexample:
classA(object):
count=0
self.name=name
defset_name(self,name):
print'settingname:%s'%name
self.name=name
defget_name(self):
print'gettingname:%s'%self.name
returnself.name
objname=property(get_name,set_name)
deftest():
a=A('apple')
print'name:%s'%a.objname
a.objname='banana'
Page45
print'name:%s'%a.objname
test()
gettingname:apple
name:apple
settingname:banana
gettingname:banana
name:banana
Notes:
Theclassinheritsfromclassobject,whichmakesitanewstyleclass.
Whenavalueisassignedtoaproperty,thesettermethodiscalled.
Whenthevalueofapropertyisaccessed,thegettermethodiscalled.
Youcanalsodefineadeletemethodandadocumentationattributeforaproperty.
Formoreinformation,visit2.1BuiltinFunctionsandlookforproperty.
1.7.3Modules
YoucanuseamoduletoorganizeanumberofPythondefinitionsinasinglefile.A
definitioncanbeafunction,aclass,oravariablecontaininganyPythonobject.Hereis
anexample:
#python_101_module_simple.py
"""
Thissimplemodulecontainsdefinitionsofaclassandseveral
functions.
"""
LABEL='=====Testingasimplemodule====='
classPerson:
"""Sampleofasimpleclassdefinition.
"""
def__init__(self,name,description):
self.name=name
self.description=description
defshow(self):
print'Personname:%sdescription:%s'%(self.name,
self.description)
deftest(msg,count):
"""Asampleofasimplefunction.
"""
foridxinrange(count):
print'%s%d'%(msg,idx)
deftestDefaultArgs(arg1='default1',arg2='default2'):
"""Afunctionwithdefaultarguments.
Page46
"""
print'arg1:',arg1
print'arg2:',arg2
deftestArgLists(*args,**kwargs):
"""
Afunctionwhichreferencestheargumentlistandkeyword
arguments.
"""
print'args:',args
defmain():
"""
Atestharnessforthismodule.
"""
printLABEL
person=Person('Herman','Acuteguy')
person.show()
print'='*30
test('Test#',4)
print'='*30
testDefaultArgs('Explicitvalue')
print'='*30
testArgLists('aaa','bbb',arg1='ccc',arg2='ddd')
main()
=====Testingasimplemodule=====
Personname:Hermandescription:Acuteguy
==============================
Test#0
Test#1
Test#2
Test#3
==============================
arg1:Explicitvalue
arg2:default2
==============================
args:('aaa','bbb')
Comments:
Thestringdefinitionsatthebeginningofeachofthemodule,classdefinitions,and
functiondefinitionsserveasdocumentationfortheseitems.Youcanshowthis
documentationwiththefollowingfromthecommandline:
$pydocpython_101_module_simple
Orthis,fromthePythoninteractiveprompt:
Page47
>>>importpython_101_module_simple
>>>help(python_101_module_simple)
Itiscommonanditisagoodpracticetoincludeatestharnessforthemoduleattheend
ofthesourcefile.Notethatthetest:
willbetrueonlywhenthefileisrun(e.g.fromthecommandlinewithsomethinglike:
"$pythonpython_101_module_simple.py
butnotwhenthemoduleisimported.
Rememberthatthecodeinamoduleisonlyevaluatedthefirsttimeitisimportedina
program.So,forexample,changethevalueofaglobalvariableinamodulemightcause
behaviorthatusersofthemodulemightnotexpect.
Constants,ontheotherhand,aresafe.Aconstant,inPython,isavariablewhosevalueis
initializedbutnotchanged.AnexampleisLABEL,above.
1.7.4Packages
Apackageisawaytoorganizeanumberofmodulestogetherasaunit.Pythonpackages
canalsocontainotherpackages.
Togiveusanexampletotalkabout,considerthefollowpackagestructure:
package_example/
package_example/__init__.py
package_example/module1.py
package_example/module2.py
package_example/A.py
package_example/B.py
And,herearethecontents:
__init__.py:
#__init__.py
#Exposedefinitionsfrommodulesinthispackage.
frommodule1importclass1
frommodule2importclass2
module1.py:
#module1.py
classclass1:
def__init__(self):
self.description='class#1'
defshow(self):
printself.description
module2.py:
Page48
#module2.py
classclass2:
def__init__(self):
self.description='class#2'
defshow(self):
printself.description
A.py:
#A.py
importB
B.py:
#B.py
deffunction_b():
print'Hellofromfunction_b'
InordertobeusedasaPythonpackage(e.g.sothatmodulescanbeimportedfromit)a
directorymustcontainafilewhosenameis__init__.py.Thecodeinthismoduleis
evaluatedthefirsttimeamoduleisimportedfromthepackage.
Inordertoimportmodulesfromapackage,youmayeitheraddthepackagedirectoryto
sys.pathor,iftheparentdirectoryisonsys.path,usedotnotationtoexplicitlyspecifythe
path.Inourexample,youmightuse:"importpackage_example.module1".
Amoduleinapackagecanimportanothermodulefromthesamepackagedirectly
withoutusingthepathtothepackage.Forexample,themoduleAinoursamplepackage
package_examplecanimportmoduleBinthesamepackagewith"importB".ModuleA
doesnotneedtouse"importpackage_example.B".
Youcanfindadditionalinformationonpackagesat
http://www.python.org/doc/essays/packages.html.
Suggestedtechniques:
Inthe__init__.pyfile,importandmakeavailableobjectsdefinedinmodulesinthe
package.Oursamplepackagepackage_exampledoesthis.Then,youcanusefrom
package_exampleimport*toimportthepackageanditscontents.Forexample:
>>>frompackage_exampleimport*
>>>dir()
['__builtins__','__doc__','__file__','__name__',
'atexit','class1','class2','module1','module2',
'readline','rlcompleter','sl','sys']
>>>
>>>c1=class1()
>>>c2=class2()
>>>c1.show()
class#1
>>>c2.show()
class#2
Afewadditionalnotes:
Page49
WithPython2.3,youcancollectthemodulesinapackageintoaZipfilebyusing
PyZipFilefromthePythonstandardlibrary.See
http://www.python.org/doc/current/lib/pyzipfileobjects.html.
>>>importzipfile
>>>a=zipfile.PyZipFile('mypackage.zip','w',
zipfile.ZIP_DEFLATED)
>>>a.writepy('Examples')
>>>a.close()
Thenyoucanimportandusethisarchivebyinsertingitspathinsys.path.Inthe
followingexample,class_basic_1isamodulewithinpackagemypackage:
>>>importsys
>>>sys.path.insert(0,
'/w2/Txt/Training/mypackage.zip')
>>>importclass_basic_1
Basicname:Apricot
>>>obj=class_basic_1.Basic('Wilma')
>>>obj.show()
Basicname:Wilma
1.8AcknowledgementsandThanks
ThankstotheimplementorsofPythonforproducinganexceptionallyusableand
enjoyableprogramminglanguage.
1.9SeeAlso
ThemainPythonWebSitehttp://www.python.orgformoreinformationon
Python.
ThePythondocumentationpagehttp://www.python.org/doc/forlotsof
documentationonPython.
Dave'sWebSitehttp://www.rexx.com/~dkuhlmanformoresoftwareand
informationonusingPythonforXMLandtheWeb.
Page50
2Part2AdvancedPython
2.1IntroductionPython201(Slightly)AdvancedPythonTopics
Thisdocumentisintendedasnotesforacourseon(slightly)advancedPythontopics.
2.2RegularExpressions
Formorehelponregularexpressions,see:
reRegularexpressionoperationshttp://docs.python.org/library/re.html
RegularExpressionHOWTOhttp://docs.python.org/howto/regex.html
2.2.1Definingregularexpressions
Aregularexpressionpatternisasequenceofcharactersthatwillmatchsequencesof
charactersinatarget.
Thepatternsorregularexpressionscanbedefinedasfollows:
Literalcharactersmustmatchexactly.Forexample,"a"matches"a".
Concatenatedpatternsmatchconcatenatedtargets.Forexample,"ab"("a"
followedby"b")matches"ab".
Alternatepatterns(separatedbyaverticalbar)matcheitherofthealternative
patterns.Forexample,"(aaa)|(bbb)"willmatcheither"aaa"or"bbb".
Repeatingandoptionalitems:
"abc*"matches"ab"followedbyzeroormoreoccurancesof"c",forexample,
"ab","abc","abcc",etc.
"abc+"matches"ab"followedbyoneormoreoccurancesof"c",forexample,
"abc","abcc",etc,butnot"ab".
"abc?"matches"ab"followedbyzerooroneoccurancesof"c",forexample,
"ab"or"abc".
SetsofcharactersCharactersandsequencesofcharactersinsquarebrackets
formaset;asetmatchesanycharacterinthesetorrange.Forexample,"[abc]"
matches"a"or"b"or"c".And,forexample,"[_az09]"matchesanunderscore
oranylowercaseletteroranydigit.
GroupsParenthesesindicateagroupwithapattern.Forexample,"ab(cd)*ef"is
apatternthatmatches"ab"followedbyanynumberofoccurancesof"cd"
followedby"ef",forexample,"abef","abcdef","abcdcdef",etc.
Therearespecialnamesforsomesetsofcharacters,forexample"\d"(anydigit),
"\w"(anyalphanumericcharacter),"\W"(anynonalphanumericcharacter),etc.
Moremoreinformation,seePythonLibraryReference:RegularExpression
Page51
Syntaxhttp://docs.python.org/library/re.html#regularexpressionsyntax
Becauseoftheuseofbackslashesinpatterns,youareusuallybetteroffdefiningregular
expressionswithrawstrings,e.g.r"abc".
2.2.2Compilingregularexpressions
Whenaregularexpressionistobeusedmorethanonce,youshouldconsidercompiling
it.Forexample:
importsys,re
pat=re.compile('aa[bc]*dd')
while1:
line=raw_input('Enteraline("q"toquit):')
ifline=='q':
break
ifpat.search(line):
print'matched:',line
else:
print'nomatch:',line
Comments:
Weimportmodulereinordertouseregularexpresions.
re.compile()compilesaregularexpressionsothatwecanreusethe
compiledregularexpressionwithoutcompilingitrepeatedly.
2.2.3Usingregularexpressions
Usematch()tomatchatthebeginningofastring(ornotatall).
Usesearch()tosearchastringandmatchthefirststringfromtheleft.
Herearesomeexamples:
>>>importre
>>>pat=re.compile('aa[09]*bb')
>>>x=pat.match('aa1234bbccddee')
>>>x
<_sre.SRE_Matchobjectat0x401e9608>
>>>x=pat.match('xxxxaa1234bbccddee')
>>>x
>>>type(x)
<type'NoneType'>
>>>x=pat.search('xxxxaa1234bbccddee')
>>>x
Notes:
Whenamatchorsearchissuccessful,itreturnsamatchobject.Whenitfails,it
returnsNone.
Page52
Youcanalsocallthecorrespondingfunctionsmatchandsearchintheremodule,
e.g.:
>>>x=re.search(pat,'xxxxaa1234bbccddee')
>>>x
Foralistoffunctionsintheremodule,seeModuleContents
http://docs.python.org/library/re.html#modulecontents.
2.2.4Usingmatchobjectstoextractavalue
Matchobjectsenableyoutoextractmatchedsubstringsafterperformingamatch.A
matchobjectisreturnedbysuccessfulmatch.Thepartofthetargetavailableinthematch
objectistheportionmatchedbygroupsinthepattern,thatistheportionofthepattern
insideparentheses.Forexample:
In[69]:mo=re.search(r'height:(\d*)width:(\d*)','height:123
width:456')
In[70]:mo.groups()
Out[70]:('123','456')
Hereisanotherexample:
importsys,re
Targets=[
'Thereare<<25>>sparrows.',
'Isee<<15>>finches.',
'Thereisnothinghere.',
]
deftest():
pat=re.compile('<<([09]*)>>')
forlineinTargets:
mo=pat.search(line)
ifmo:
value=mo.group(1)
print'value:%s'%value
else:
print'nomatch'
test()
Whenweruntheabove,itprintsoutthefollowing:
value:25
value:15
nomatch
Explanation:
Intheregularexpression,putparenthesesaroundtheportionoftheregular
expressionthatwillmatchwhatyouwanttoextract.Eachpairofparentheses
marksoffagroup.
Page53
Afterthesearch,checktodetermineiftherewasasuccessfulmatchbychecking
foramatchingobject."pat.search(line)"returnsNoneifthesearchfails.
Ifyouspecifymorethanonegroupinyourregularexpression(morethatonepair
ofparentheses),thenyoucanuse"value=mo.group(N)"toextractthevalue
matchedbytheNthgroupfromthematchingobject."value=mo.group(1)"
returnsthefirstextractedvalue;"value=mo.group(2)"returnsthesecond;etc.An
argumentof0returnsthestringmatchedbytheentireregularexpression.
Inaddition,youcan:
Use"values=mo.groups()"togetatuplecontainingthestringsmatchedbyall
groups.
Use"mo.expand()"tointerpolatethegroupvaluesintoastring.Forexample,
"mo.expand(r'value1:\1value2:\2')"insertsthevaluesofthefirstandsecond
groupintoastring.Ifthefirstgroupmatched"aaa"andthesecondmatched
"bbb",thenthisexamplewouldproduce"value1:aaavalue2:bbb".Forexample:
In[76]:mo=re.search(r'h:(\d*)w:(\d*)','h:123
w:456')
In[77]:mo.expand(r'Height:\1Width:\2')
Out[77]:'Height:123Width:456'
2.2.5Extractingmultipleitems
Youcanextractmultipleitemswithasinglesearch.Hereisanexample:
importsys,re
pat=re.compile('aa([09]*)bb([09]*)cc')
while1:
ifline=='q':
break
mo=pat.search(line)
ifmo:
value1,value2=mo.group(1,2)
print'value1:%svalue2:%s'%(value1,value2)
else:
print'nomatch'
Comments:
Usemultipleparenthesizedsubstringsintheregularexpressiontoindicatethe
portions(groups)tobeextracted.
"mo.group(1,2)"returnsthevaluesofthefirstandsecondgroupinthestring
matched.
Wecouldalsohaveused"mo.groups()"toobtainatuplethatcontainsboth
values.
Yetanotheralternativewouldhavebeentousethefollowing:print
mo.expand(r'value1:\1value2:\2').
Page54
2.2.6Replacingmultipleitems
Asimplewaytoperformmultiplereplacementsusingaregularexpressionistousethe
re.subn()function.Hereisanexample:
In[81]:re.subn(r'\d+','***','thereare203birdssittingin2
trees')
Out[81]:('thereare***birdssittingin***trees',2)
Formorecomplexreplacements,useafunctioninsteadofaconstantreplacementstring:
importre
defrepl_func(mo):
s1=mo.group(1)
s2='*'*len(s1)
returns2
deftest():
pat=r'(\d+)'
in_str='thereare2034birdsin21trees'
out_str,count=re.subn(pat,repl_func,in_str)
print'in:"%s"'%in_str
print'out:"%s"'%out_str
print'count:%d'%count
test()
Andwhenweruntheabove,itproduces:
in:"thereare2034birdsin21trees"
out:"thereare****birdsin**trees"
count:2
Notes:
Thereplacementfunctionreceivesoneargument,amatchobject.
There.subn()functionreturnsatuplecontainingtwovalues:(1)thestring
afterreplacementsand(2)thenumberofreplacementsperformed.
HereisanevenmorecomplexexampleYoucanlocatesubstrings(slices)ofamatch
andreplacethem:
importsys,re
pat=re.compile('aa([09]*)bb([09]*)cc')
while1:
ifline=='q':
break
mo=pat.search(line)
ifmo:
value1,value2=mo.group(1,2)
start1=mo.start(1)
Page55
end1=mo.end(1)
start2=mo.start(2)
end2=mo.end(2)
print'value1:%sstart1:%dend1:%d'%(value1,start1,
end1)
print'value2:%sstart2:%dend2:%d'%(value2,start2,
end2)
repl1=raw_input('Enterreplacement#1:')
repl2=raw_input('Enterreplacement#2:')
newline=(line[:start1]+repl1+line[end1:start2]+
repl2+line[end2:])
print'newline:%s'%newline
else:
print'nomatch'
Explanation:
Alternatively,use"mo.span(1)"insteadof"mo.start(1)"and"mo.end(1)"inorder
togetthestartandendofasubmatchinasingleoperation."mo.span(1)"returnsa
tuple:(start,end).
Puttogetheranewstringwithstringconcatenationfrompiecesoftheoriginal
stringandreplacementvalues.Youcanusestringslicestogetthesubstringsof
theoriginalstring.Inourcase,thefollowinggetsthestartofthestring,addsthe
firstreplacement,addsthemiddleoftheoriginalstring,addsthesecond
replacement,andfinally,addsthelastpartoftheoriginalstring:
newline=line[:start1]+repl1+line[end1:start2]+
repl2+line[end2:]
Youcanalsousethesubfunctionormethodtodosubstitutions.Hereisanexample:
importsys,re
pat=re.compile('[09]+')
print'Replacingdecimaldigits.'
while1:
target=raw_input('Enteratargetline("q"toquit):')
iftarget=='q':
break
repl=raw_input('Enterareplacement:')
result=pat.sub(repl,target)
print'result:%s'%result
Hereisanotherexampleoftheuseofafunctiontoinsertcalculatedreplacements.
importsys,re,string
pat=re.compile('[am]+')
defreplacer(mo):
returnstring.upper(mo.group(0))
print'Uppercasingam.'
while1:
Page56
iftarget=='q':
break
result=pat.sub(replacer,target)
Notes:
Ifthereplacementargumenttosubisafunction,thatfunctionmusttakeone
argument,amatchobject,andmustreturnthemodified(orreplacement)value.
Thematchedsubstringwillbereplacedbythevaluereturnedbythisfunction.
Inourcase,thefunctionreplacerconvertsthematchedvaluetouppercase.
Thisisalsoaconvenientuseforalambdainsteadofanamedfunction,forexample:
importsys,re,string
pat=re.compile('[am]+')
print'Uppercasingam.'
while1:
iftarget=='q':
break
result=pat.sub(
lambdamo:string.upper(mo.group(0)),
target)
2.3IteratorObjects
Note1:YouwillneedasufficientlyrecentversionofPythoninordertouseiteratorsand
generators.IbelievethattheywereintroducedinPython2.2.
Note2:TheiteratorprotocolhaschangedslightlyinPythonversion3.0.
Goalsforthissection:
Learnhowtoimplementageneratorfunction,thatis,afunctionwhich,when
called,returnsaniterator.
Learnhowtoimplementaclasscontainingageneratormethod,thatis,amethod
which,whencalled,returnsaniterator.
Learntheiteratorprotocol,specificallywhatmethodsaniteratormustsupportand
whatthosemethodsmustdo.
Learnhowtoimplementaniteratorclass,thatis,aclasswhoseinstancesare
iteratorobjects.
Learnhowtoimplementrecursiveiteratorgenerators,thatis,aniteratorgenerator
whichrecursivelyproducesiteratorgenerators.
Learnthatyourimplementationofaniteratorobject(aniteratorclass)can
"refresh"itselfandlearnatleastonewaytodothis.
Definitions:
Page57
IteratorAnditeratorisanobjectthatsatisfies(implements)theiteratorprotocol.
IteratorprotocolAnobjectimplementstheiteratorprotocolifitimplementsboth
anext()andan__iter__()methodwhichsatisfytheserules:(1)the
__iter__()methodmustreturntheiterator;(2)thenext()methodshould
returnthenextitemtobeiteratedoverandwhenfinished(therearenomore
items)shouldraisetheStopIterationexception.Theiteratorprotocolis
describedatIteratorTypeshttp://docs.python.org/library/stdtypes.html#iterator
types.
IteratorclassAclassthatimplements(satisfies)theiteratorprotocol.In
particular,theclassimplementsnext()and__iter__()methodsas
describedaboveandinIteratorTypes
(Iterator)generatorfunctionAfunction(ormethod)which,whencalled,returns
aniteratorobject,thatis,anobjectthatsatisfiestheiteratorprotocol.Afunction
containingayieldstatementautomaticallybecomesagenerator.
GeneratorexpressionAnexpressionwhichproducesaniteratorobject.
Generatorexpressionshaveaformsimilartoalistcomprehension,butare
enclosedinparenthesesratherthansquarebrackets.Seeexamplebelow.
Afewadditionalbasicpoints:
Afunctionthatcontainsayieldstatementisageneratorfunction.Whencalled,it
returnsaniterator,thatis,anobjectthatprovidesnext()and__iter__()
methods.
Theiteratorprotocolisdescribedhere:PythonStandardLibrary:IteratorTypes
Aclassthatdefinesbothanext()methodanda__iter__()methodsatisfies
theiteratorprotocol.So,instancesofsuchaclasswillbeiterators.
Pythonprovidesavarietyofwaystoproduce(implement)iterators.Thissection
describesafewofthoseways.Youshouldalsolookattheiter()builtin
function,whichisdescribedinThePythonStandardLibrary:BuiltinFunctions:
iter()http://docs.python.org/library/functions.html#iter.
Aniteratorcanbeusedinaniteratorcontext,forexampleinaforstatement,ina
listcomprehension,andinageneratorexpression.Whenaniteratorisusedinan
iteratorcontext,theiteratorproducesitsvalues.
Thissectionattemptstoprovideexamplesthatillustratethegenerator/iteratorpattern.
Whyisthisimportant?
Oncemastered,itisasimple,convenient,andpowerfulprogrammingpattern.
Ithasmanyandpervasiveuses.
Ithelpstolexicallyseparatetheproducercodefromtheconsumercode.Doingso
makesiteasiertolocateproblemsandtomodifyorfixcodeinawaythatis
localizedanddoesnothaveunwantedsideeffects.
Implementingyourowniterators(andgenerators)enablesyoutodefineyourown
Page58
abstractsequences,thatis,sequenceswhosecompositionaredefinedbyyour
computationsratherthanbytheirpresenceinacontainer.Infact,youriteratorcan
calculateorretrievevaluesaseachoneisrequested.
ExamplesTheremainderofthissectionprovidesasetofexampleswhichimplement
anduseiterators.
2.3.1ExampleAgeneratorfunction
Thisfunctioncontainsayieldstatement.Therefore,whenwecallit,itproducesan
iterator:
defgenerateItems(seq):
foriteminseq:
yield'item:%s'%item
anIter=generateItems([])
print'dir(anIter):',dir(anIter)
anIter=generateItems([111,222,333])
forxinanIter:
printx
anIter=generateItems(['aaa','bbb','ccc'])
printanIter.next()
printanIter.next()
printanIter.next()
printanIter.next()
Runningthisexampleproducesthefollowingoutput:
dir(anIter):['__class__','__delattr__','__doc__',
'__getattribute__',
'__hash__','__init__','__iter__','__new__','__reduce__',
'__reduce_ex__','__repr__','__setattr__','__str__','gi_frame',
'gi_running','next']
item:111
item:222
item:333
item:aaa
item:bbb
item:ccc
File"iterator_generator.py",line14,in?
printanIter.next()
StopIteration
Notesandexplanation:
Thevaluereturnedbythecalltothegenerator(function)isaniterator.Itobeys
theiteratorprotocol.Thatis,dir(anIter)showsthatithasboth
__iter__()andnext()methods.
Becausethisobjectisaniterator,wecanuseaforstatementtoiterateoverthe
valuesreturnedbythegenerator.
Wecanalsogetitsvaluesbyrepeatedlycallingthenext()method,untilit
Page59
raisestheStopIterationexception.Thisabilitytocallthenextmethodenablesus
topasstheiteratorobjectaroundandgetvaluesatdifferentlocationsinourcode.
Oncewehaveobtainedallthevaluesfromaniterator,itis,ineffect,"empty"or
"exhausted".Theiteratorprotocol,infact,specifiesthatonceaniteratorraisesthe
StopIterationexception,itshouldcontinuetodoso.Anotherwaytosaythisis
thatthereisno"rewind"operation.But,youcancallthethegeneratorfunction
againtogeta"fresh"iterator.
Analternativeandperhapssimplerwaytocreateaninteratoristouseagenerator
expression.Thiscanbeusefulwhenyoualreadyhaveacollectionoriteratortowork
with.
Thenfollowingexampleimplementsafunctionthatreturnsageneratorobject.Theeffect
istogeneratetheobjectsinacollectionwhichexcludingitemsinasepartecollection:
DATA=[
'lemon',
'lime',
'grape',
'apple',
'pear',
'watermelon',
'canteloupe',
'honeydew',
'orange',
'grapefruit',
]
defmake_producer(collection,excludes):
gen=(itemforitemincollectionifitemnotinexcludes)
returngen
deftest():
iter1=make_producer(DATA,('apple','orange','honeydew',))
print'%s'%iter1
forfruitiniter1:
printfruit
test()
Whenrun,thisexampleproducesthefollowing:
$pythonworkbook063.py
<generatorobject<genexpr>at0x7fb3d0f1bc80>
lemon
lime
grape
pear
watermelon
canteloupe
grapefruit
Notes:
Page60
Ageneratorexpressionlooksalmostlikealistcomprehension,butissurrounded
byparenthesesratherthansquarebrackets.Formoreonlistcomprehensionssee
sectionExampleAlistcomprehension.
Themake_producerfunctionreturnstheobjectproducedbythegenerator
expression.
2.3.2ExampleAclasscontainingageneratormethod
Eachtimethismethodiscalled,itproducesa(new)iteratorobject.Thismethodis
analogoustotheiterkeysanditervaluesmethodsinthedictionarybuiltinobject:
#
#Aclassthatprovidesaniteratorgeneratormethod.
#
classNode:
def__init__(self,name='<noname>',value='<novalue>',
children=None):
self.name=name
self.value=value
self.children=children
ifchildrenisNone:
self.children=[]
else:
defset_name(self,name):self.name=name
defget_name(self):returnself.name
defset_value(self,value):self.value=value
defget_value(self):returnself.value
defiterchildren(self):
forchildinself.children:
yieldchild
#
#Printinformationonthisnodeandwalkoverallchildrenand
#grandchildren...
defwalk(self,level=0):
print'%sname:%svalue:%s'%(
get_filler(level),self.get_name(),self.get_value(),)
forchildinself.iterchildren():
child.walk(level+1)
#
#Anfunctionthatistheequivalentofthewalk()methodin
#classNode.
#
defwalk(node,level=0):
print'%sname:%svalue:%s'%(
get_filler(level),node.get_name(),node.get_value(),)
forchildinnode.iterchildren():
walk(child,level+1)
defget_filler(level):
return''*level
Page61
deftest():
a7=Node('gilbert','777')
a6=Node('fred','666')
a5=Node('ellie','555')
a4=Node('daniel','444')
a3=Node('carl','333',[a4,a5])
a2=Node('bill','222',[a6,a7])
a1=Node('alice','111',[a2,a3])
#Usethewalkmethodtowalktheentiretree.
print'Usingthemethod:'
a1.walk()
print'='*30
#Usethewalkfunctiontowalktheentiretree.
print'Usingthefunction:'
walk(a1)
test()
Usingthemethod:
name:alicevalue:111
name:billvalue:222
name:fredvalue:666
name:gilbertvalue:777
name:carlvalue:333
name:danielvalue:444
name:ellievalue:555
==============================
Usingthefunction:
name:alicevalue:111
name:billvalue:222
name:fredvalue:666
name:gilbertvalue:777
name:carlvalue:333
name:danielvalue:444
name:ellievalue:555
Thisclasscontainsamethoditerchildrenwhich,whencalled,returnsaniterator.
Theyieldstatementinthemethoditerchildrenmakesitintoagenerator.
Theyieldstatementreturnsoneitemeachtimeitisreached.Thenexttimethe
iteratorobjectis"called"itresumesimmediatelyaftertheyieldstatement.
Afunctionmayhaveanynumberofyieldstatements.
Aforstatementwilliterateoveralltheitemsproducedbyaniteratorobject.
Thisexampleshowstwowaystousethegenerator,specifically:(1)thewalk
methodintheclassNodeand(2)thewalkfunction.Bothcallthegenerator
iterchildrenandbothdoprettymuchthesamething.
2.3.3ExampleAniteratorclass
Thisclassimplementstheiteratorprotocol.Therefore,instancesofthisclassareiterators.
Page62
Thepresenceofthenext()and__iter__()methodsmeansthatthisclass
implementstheiteratorprotocolandmakesinstancesofthisclassiterators.
Notethatwhenaniteratoris"exhausted"it,normally,cannotbereusedtoiterateoverthe
sequence.However,inthisexample,weprovidearefreshmethodwhichenablesusto
"rewind"andreusetheiteratorinstance:
#
#Aniteratorclassthatdoes*not*use``yield``.
#Thisiteratorproduceseveryotheriteminasequence.
#
classIteratorExample:
def__init__(self,seq):
self.seq=seq
self.idx=0
defnext(self):
self.idx+=1
ifself.idx>=len(self.seq):
raiseStopIteration
value=self.seq[self.idx]
self.idx+=1
returnvalue
def__iter__(self):
returnself
defrefresh(self):
self.idx=0
deftest_iteratorexample():
a=IteratorExample('edcba')
forxina:
printx
print''
a.refresh()
forxina:
printx
print'='*30
a=IteratorExample('abcde')
try:
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
exceptStopIteration,e:
print'stopping',e
d
b
d
b
==============================
Page63
b
d
stopping
Thenextmethodmustkeeptrackofwhereitisandwhatitemitshouldproduce
next.
Alert:TheiteratorprotocolhaschangedslightlyinPython3.0.Inparticular,the
next()methodhasbeenrenamedto__next__().See:PythonStandard
Library:IteratorTypeshttp://docs.python.org/3.0/library/stdtypes.html#iterator
types.
2.3.4ExampleAniteratorclassthatusesyield
Theremaybetimeswhenthenextmethodiseasierandmorestraightforwardto
implementusingyield.Ifso,thenthisclassmightserveasanmodel.Ifyoudonotfeel
theneedtodothis,thenyoushouldignorethisexample:
#
#Aniteratorclassthatuses``yield``.
#Thisiteratorproduceseveryotheriteminasequence.
#
classYieldIteratorExample:
def__init__(self,seq):
self.seq=seq
self.iterator=self._next()
self.next=self.iterator.next
def_next(self):
flag=0
forxinself.seq:
ifflag:
flag=0
yieldx
else:
flag=1
def__iter__(self):
returnself.iterator
defrefresh(self):
deftest_yielditeratorexample():
a=YieldIteratorExample('edcba')
forxina:
printx
print''
a.refresh()
forxina:
printx
print'='*30
a=YieldIteratorExample('abcde')
try:
Page64
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
exceptStopIteration,e:
print'stopping',e
test_yielditeratorexample()
d
b
d
b
==============================
b
d
stopping
Becausethe_nextmethodusesyield,callingit(actually,callingtheiterator
objectitproduces)inaniteratorcontextcausesittobe"resumed"immediately
aftertheyieldstatement.Thisreducesbookkeepingabit.
However,withthisstyle,wemustexplicitlyproduceaniterator.Wedothisby
callingthe_nextmethod,whichcontainsayieldstatement,andisthereforea
generator.Thefollowingcodeinourconstructor(__init__)completestheset
upofourclassasaniteratorclass:
Rememberthatweneedboth__iter__()andnext()methodsin
orderDictionarytosatisfytheiteratorprotocol.The__iter__()methodis
alreadythereandtheabovecodeintheconstructorcreatesthenext()method.
2.3.5ExampleAlistcomprehension
Alistcomprehensionlooksabitlikeaniterator,butitproducesalist.See:ThePython
LanguageReference:Listdisplays
http://docs.python.org/reference/expressions.html#listdisplaysformoreonlist
comprehensions.
Hereisanexample:
In[4]:deff(x):
...:returnx*3
...:
In[5]:list1=[11,22,33]
Page65
In[6]:list2=[f(x)forxinlist1]
In[7]:printlist2
[33,66,99]
2.3.6ExampleAgeneratorexpression
Ageneratorexpressionlooksquitesimilartoalistcomprehension,butisenclosedin
parenthesesratherthansquarebrackets.Unlikealistcomprehension,agenerator
expressiondoesnotproducealist;itproducesangeneratorobject.Ageneratorobjectis
aniterator.
Formoreongeneratorexpressions,seeThePythonLanguageReference:Generator
expressionshttp://docs.python.org/reference/expressions.html#generatorexpressions.
Thefollowingexampleusesageneratorexpressiontoproduceaniterator:
mylist=range(10)
deff(x):
returnx*3
genexpr=(f(x)forxinmylist)
forxingenexpr:
printx
Thegeneratorexpression(f(x)forxinmylist)producesaniteratorobject.
Noticethatwecanusetheiteratorobjectlaterinourcode,cansaveitinadata
structure,andcanpassittoafunction.
2.4UnitTests
UnittestandthePythonunittestframeworkprovideaconvenientwaytodefineandrun
teststhatensurethataPythonapplicationproducesspecifiedresults.
Thissection,whileitwillnotattempttoexplaineverythingabouttheunittestframework,
willprovideexamplesofseveralstraightforwardwaystoconstructandruntests.
Someassumptions:
Wearegoingtodevelopasoftwareprojectincrementally.Wewillnotimplement
andreleaseallatonce.Therefore,eachtimeweaddtoourexistingcodebase,we
needawaytoverifythatouradditions(andfixes)havenotcausednewproblems
inoldcode.
Addingnewcodetoexistingcodewillcauseproblems.Weneedtobeableto
check/testforthoseproblemsateachstep.
Asweaddcode,weneedtobeabletoaddtestsforthatnewcode,too.
Page66
2.4.1Definingunittests
2.4.1.1Createatestclass.
Inthetestclass,implementanumberofmethodstoperformyourtests.Nameyourtest
methodswiththeprefix"test".Hereisanexample:
classMyTest:
deftest_one(self):
#sometestcode
pass
deftest_two(self):
#sometestcode
pass
Createatestharness.Hereisanexample:
#makethetestsuite.
defsuite():
loader=unittest.TestLoader()
testsuite=loader.loadTestsFromTestCase(MyTest)
returntestsuite
#Makethetestsuite;runthetests.
deftest():
testsuite=suite()
runner=unittest.TextTestRunner(sys.stdout,verbosity=2)
result=runner.run(testsuite)
Hereisamorecompleteexample:
importsys,StringIO,string
importunittest
importwebserv_example_heavy_sub
#Acomparisonfunctionforcaseinsenstivesorting.
defmycmpfunc(arg1,arg2):
returncmp(string.lower(arg1),string.lower(arg2))
classXmlTest(unittest.TestCase):
deftest_import_export1(self):
inFile=file('test1_in.xml','r')
inContent=inFile.read()
inFile.close()
doc=webserv_example_heavy_sub.parseString(inContent)
outFile=StringIO.StringIO()
outFile.write('<?xmlversion="1.0"?>\n')
doc.export(outFile,0)
outContent=outFile.getvalue()
outFile.close()
self.failUnless(inContent==outContent)
#makethetestsuite.
defsuite():
Page67
loader=unittest.TestLoader()
#Changethetestmethodprefix:test>trial.
#loader.testMethodPrefix='trial'
#Changethecomparisonfunctionthatdeterminestheorderof
tests.
#loader.sortTestMethodsUsing=mycmpfunc
testsuite=loader.loadTestsFromTestCase(XmlTest)
returntestsuite
#Makethetestsuite;runthetests.
deftest_main():
testsuite=suite()
runner=unittest.TextTestRunner(sys.stdout,verbosity=2)
result=runner.run(testsuite)
if__name__=="__main__":
test_main()
Runningtheabovescriptproducesthefollowingoutput:
test_import_export(__main__.XmlTest)...ok
Ran1testin0.035s
OK
Afewnotesonthisexample:
Thisexampleteststheabilitytoparseanxmldocumenttest1_in.xmlandexport
thatdocumentbacktoXML.ThetestsucceedsiftheinputXMLdocumentand
theexportedXMLdocumentarethesame.
ThecodewhichisbeingtestedparsesanXMLdocumentreturnedbyarequestto
AmazonWebservices.YoucanlearnmoreaboutAmazonWebservicesat:
http://www.amazon.com/webservices.ThiscodewasgeneratedfromanXML
SchemadocumentbygenerateDS.py.Soweareineffect,testinggenerateDS.py.
YoucanfindgenerateDS.pyat:http://www.rexx.com/~dkuhlman/#generateDS.
Testingforsuccess/failureandreportingfailuresUsethemethodslistedat
http://www.python.org/doc/current/lib/testcaseobjects.htmltotestforandreport
successandfailure.Inourexample,weused"self.failUnless(inContent==
outContent)"toensurethatthecontentweparsedandthecontentthatwe
exportedwerethesame.
Addadditionaltestsbyaddingmethodswhosenameshavetheprefix"test".If
youpreferadifferentprefixfortestsnames,addsomethinglikethefollowingto
theabovescript:
loader.testMethodPrefix='trial'
Bydefault,thetestsarerunintheorderoftheirnamessortedbythecmp
function.So,ifneeded,youcancontroltheorderofexecutionoftestsby
selectingtheirnames,forexample,usingnamesliketest_1_checkderef,
Page68
test_2_checkcalc,etc.Or,youcanchangethecomparisonfunctionbyadding
somethinglikethefollowingtotheabovescript:
loader.sortTestMethodsUsing=mycmpfunc
Asabitofmotivationforcreatingandusingunittests,whiledevelopingthisexample,I
discoveredseveralerrors(ormaybe"specialfeatures")ingenerateDS.py.
2.5ExtendingandembeddingPython
2.5.1Introductionandconcepts
Extendingvs.embeddingTheyaredifferentbutrelated:
ExtendingPythonmeanstoimplementanextensionmoduleoranextensiontype.
AnextensionmodulecreatesanewPythonmodulewhichisimplementedin
C/C++.FromPythoncode,anextensionmoduleappearstobejustlikeamodule
implementedinPythoncode.AnextensiontypecreatesanewPython(builtin)
typewhichisimplementedinC/C++.FromPythoncode,anextensiontype
appearstobejustlikeabuiltintype.
EmbeddingPython,bycontrast,istoputthePythoninterpreterwithinan
application(i.e.linkitin)sothattheapplicationcanrunPythonscripts.The
scriptscanbeexecutedortriggeredinavarietyofways,e.g.theycanbeboundto
keysonthekeyboardortomenuitems,theycanbetriggeredbyexternalevents,
etc.Usually,inordertomaketheembeddedPythoninterpreteruseful,Pythonis
alsoextendedwithfunctionsfromtheembeddingapplication,sothatthescripts
cancallfunctionsthatareimplementedbytheembeddingC/C++application.
DocumentationThetwoimportantsourcesforinformationaboutextendingand
embeddingarethefollowing:
ExtendingandEmbeddingthePythonInterpreter
http://www.python.org/doc/current/ext/ext.html
Python/CAPIReferenceManual
http://www.python.org/doc/current/api/api.html
Typesofextensions:
ExtensionmodulesFromthePythonside,itappearstobeaPythonmodule.
Usuallyitexportsfunctions.
ExtensiontypesUsedtoimplementanewPythondatatype.
ExtensionclassesFromthePythonside,itappearstobeaclass.
ToolsThereareseveraltoolsthatsupportthedevelopmentofPythonextensions:
SWIGLearnaboutSWIGat:http://www.swig.org
PyrexLearnaboutPyrexat:
http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
ThereisalsoCython,whichseemstobeanadvancedversionof,oratleastan
Page69
alternativetoPyrex.See:CythonCExtensionsforPython
http://www.cython.org/
2.5.2Extensionmodules
WritinganextensionmodulebyhandWhattodo:
Createthe"init"functionThenameofthisfunctionmustbe"init"followedby
thenameofthemodule.Everyextensionmodulemusthavesuchafunction.
CreatethefunctiontableThistablemapsfunctionnames(referencedfrom
Pythoncode)tofunctionpointers(implementedinC/C++).
Implementeachwrapperfunction.
ImplementingawrapperfunctionWhattodo:
3. CapturetheargumentswithPyArg_ParseTuple.Theformatstringspecifieshow
argumentsaretobeconvertedandcaptured.See1.7ExtractingParametersin
ExtensionFunctions.Herearesomeofthemostcommonlyusedtypes:
Use"i","s","f",etctoconvertandcapturesimpletypessuchasintegers,
strings,floats,etc.
Use"O"togetapointertoPython"complex"typessuchaslists,tuples,
dictionaries,etc.
Useitemsinparenthesestocaptureandunpacksequences(e.g.listsand
tuples)offixedlength.Example:
if(!PyArg_ParseTuple(args,"(ii)(ii)",&x,&y,
&width,&height))
{
returnNULL;
}/*if*/
Asamplecallmightbe:
lowerLeft=(x1,y1)
extent=(width1,height1)
scan(lowerLeft,extent)
Use":aName"(colon)attheendoftheformatstringtoprovideafunction
nameforerrormessages.Example:
if(!PyArg_ParseTuple(args,"O:setContentHandler",
&pythonInstance))
{
returnNULL;
}/*if*/
Use";anerrormessage"(semicolon)attheendoftheformatstringtoprovide
astringthatreplacesthedefaulterrormessage.
Docsareavailableat:http://www.python.org/doc/current/ext/parseTuple.html.
4. Writethelogic.
5. HandleerrorsandexceptionsYouwillneedtounderstandhowto(1)clearing
errorsandexceptionsand(2)Raiseerrors(exceptions).
ManyfunctionsinthePythonCAPIraiseexceptions.Youwillneedtocheck
forandcleartheseexceptions.Hereisanexample:
Page70
char*message;
intmessageNo;
message=NULL;
messageNo=1;
/*Istheargumentastring?
*/
if(!PyArg_ParseTuple(args,"s",&message))
{
/*It'snotastring.Cleartheerror.
*Thentrytogetamessagenumber(an
integer).
*/
PyErr_Clear();
if(!PyArg_ParseTuple(args,"i",&messageNo))
{
o
o
o
YoucanalsoraiseexceptionsinyourCcodethatcanbecaught(ina
"try:except:"block)backinthecallingPythoncode.Hereisanexample:
if(n==0)
{
PyErr_SetString(PyExc_ValueError,"Valuemust
notbezero");
returnNULL;
}
SeeInclude/pyerrors.hinthePythonsourcedistributionformore
exception/errortypes.
And,youcantestwhetherafunctioninthePythonCAPIthatyouhavecalled
hasraisedanexception.Forexample:
if(PyErr_Occurred())
{
/*Anexceptionwasraised.
*Dosomethingaboutit.
*/
o
o
o
Formoredocumentationonerrorsandexceptions,see:
http://www.python.org/doc/current/api/exceptionHandling.html.
6. Createandreturnavalue:
ForeachbuiltinPythontypethereisasetofAPIfunctionstocreateand
manipulateit.Seethe"Python/CAPIReferenceManual"foradescriptionof
thesefunctions.Forexample,see:
http://www.python.org/doc/current/api/intObjects.html
http://www.python.org/doc/current/api/stringObjects.html
http://www.python.org/doc/current/api/tupleObjects.html
http://www.python.org/doc/current/api/listObjects.html
http://www.python.org/doc/current/api/dictObjects.html
Page71
Etc.
ThereferencecountYouwillneedtofollowPython'srulesforreference
countingthatPythonusestogarbagecollectobjects.Youcanlearnabout
theserulesathttp://www.python.org/doc/current/ext/refcounts.html.Youwill
notwantPythontogarbagecollectobjectsthatyoucreatetooearlyortoolate.
WithrespecttoPythonobjectscreatedwiththeabovefunctions,thesenew
objectsareownedandmaybepassedbacktoPythoncode.However,there
aresituationswhereyourC/C++codewillnotautomaticallyownareference,
forexamplewhenyouextractanobjectfromacontainer(alist,tuple,
dictionary,etc).Inthesecasesyoushouldincrementthereferencecountwith
Py_INCREF.
2.5.3SWIG
Note:OurdiscussionandexamplesareforSWIGversion1.3
SWIGwilloftenenableyoutogeneratewrappersforfunctionsinanexistingCfunction
library.SWIGdoesnotunderstandeverythinginCheaderfiles.Butitdoesafairly
impressivejob.Youshouldtryitfirstbeforeresortingtothehardworkofwriting
wrappersbyhand.
MoreinformationonSWIGisathttp://www.swig.org.
Herearesomestepsthatyoucanfollow:
1. CreateaninterfacefileEvenwhenyouarewrappingfunctionsdefinedinan
existingheaderfile,creatinganinterfacefileisagoodidea.Includeyourexisting
headerfileintoit,thenaddwhateverelseyouneed.Hereisanextremelysimple
exampleofaSWIGinterfacefile:
%moduleMyLibrary
%{
#include"MyLibrary.h"
%}
%include"MyLibrary.h"
Comments:
The"%{"and"%}"bracketsaredirectivestoSWIG.Theysay:"Addthecode
betweenthesebracketstothegeneratedwrapperfilewithoutprocessingit.
The"%include"statementsays:"Copythefileintotheinterfacefilehere.In
effect,youareaskingSWIGtogeneratewrappersforallthefunctionsinthis
headerfile.Ifyouwantwrappersforonlysomeofthefunctionsinaheader
file,thencopyorreproducefunctiondeclarationsforthedesiredfunctions
here.Anexample:
%moduleMyLibrary
%{
Page72
#include"MyLibrary.h"
%}
intcalcArea(intwidth,intheight);
intcalcVolume(intradius);
Thisexamplewillgeneratewrappersforonlytwofunctions.
YoucanfindmoreinformationaboutthedirectivesthatareusedinSWIG
interfacefilesintheSWIGUserManual,inparticularat:
http://www.swig.org/Doc1.3/Preprocessor.html
http://www.swig.org/Doc1.3/Python.html
2. Generatethewrappers:
swigpythonMyLibrary.i
3. Compileandlinkthelibrary.OnLinux,youcanusesomethinglikethefollowing:
gcccMyLibrary.c
gcccI/usr/local/include/python2.3MyLibrary_wrap.c
gccsharedMyLibrary.oMyLibrary_wrap.oo
_MyLibrary.so
Notethatweproduceasharedlibrarywhosenameisthemodulenameprefixed
withanunderscore.SWIGalsogeneratesa.pyfile,withouttheleading
underscore,whichwewillimportfromourPythoncodeandwhich,inturn,
importsthesharedlibrary.
4. Usetheextensionmoduleinyourpythoncode:
Python2.3b1(#1,Apr252003,20:36:09)
[GCC2.95.420011002(Debianprerelease)]onlinux2
Type"help","copyright","credits"or"license"for
moreinformation.
>>>importMyLibrary
>>>MyLibrary.calcArea(4.0,5.0)
20.0
Hereisamakefilethatwillexecuteswigtogeneratewrappers,thencompileandlinkthe
extension.
CFLAGS=I/usr/local/include/python2.3
all:_MyLibrary.so
_MyLibrary.so:MyLibrary.oMyLibrary_wrap.o
gccsharedMyLibrary.oMyLibrary_wrap.oo_MyLibrary.so
MyLibrary.o:MyLibrary.c
gcccMyLibrary.coMyLibrary.o
MyLibrary_wrap.o:MyLibrary_wrap.c
gccc${CFLAGS}MyLibrary_wrap.coMyLibrary_wrap.o
MyLibrary_wrap.c:MyLibrary.i
clean:
rmfMyLibrary.pyMyLibrary.oMyLibrary_wrap.c
Page73
MyLibrary_wrap.o_MyLibrary.so
Hereisanexampleofrunningthismakefile:
$makefMyLibrary_makefileclean
rmfMyLibrary.pyMyLibrary.oMyLibrary_wrap.c\
MyLibrary_wrap.o_MyLibrary.so
$makefMyLibrary_makefile
gcccMyLibrary.coMyLibrary.o
gcccI/usr/local/include/python2.3MyLibrary_wrap.co
MyLibrary_wrap.o
gccsharedMyLibrary.oMyLibrary_wrap.oo_MyLibrary.so
And,hereareCsourcefilesthatcanbeusedinourexample.
MyLibrary.h:
/*MyLibrary.h
*/
floatcalcArea(floatwidth,floatheight);
floatcalcVolume(floatradius);
intgetVersion();
intgetMode();
MyLibrary.c:
/*MyLibrary.c
*/
floatcalcArea(floatwidth,floatheight)
{
return(width*height);
}
floatcalcVolume(floatradius)
{
return(3.14*radius*radius);
}
intgetVersion()
{
return123;
}
intgetMode()
{
return1;
}
Page74
2.5.4Pyrex
PyrexisausefultoolforwritingPythonextensions.BecausethePyrexlanguageis
similartoPython,writingextensionsinPyrexiseasierthandoingsoinC.Cython
appearstobetheanewerversionofPyrex.
MoreinformationisonPyrexandCythonisat:
Pyrexhttp://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
CythonCExtensionsforPythonhttp://www.cython.org/
HereisasimplefunctiondefinitioninPyrex:
#python_201_pyrex_string.pyx
importstring
defformatString(objects1,objects2):
s1=string.strip(s1)
s2=string.strip(s2)
s3='<<%s||%s>>'%(s1,s2)
s4=s3*4
returns4
And,hereisamakefile:
CFLAGS=DNDEBUGO3WallWstrictprototypesfPIC\
I/usr/local/include/python2.3
all:python_201_pyrex_string.so
python_201_pyrex_string.so:python_201_pyrex_string.o
gccsharedpython_201_pyrex_string.oo
python_201_pyrex_string.so
python_201_pyrex_string.o:python_201_pyrex_string.c
gccc${CFLAGS}python_201_pyrex_string.co
python_201_pyrex_string.o
python_201_pyrex_string.c:python_201_pyrex_string.pyx
pyrexcpython_201_pyrex_string.pyx
clean:
rmfpython_201_pyrex_string.sopython_201_pyrex_string.o\
python_201_pyrex_string.c
Hereisanotherexample.Inthisone,onefunctioninthe.pyxfilecallsanother.Hereis
theimplementationfile:
#python_201_pyrex_primes.pyx
defshowPrimes(intkmax):
plist=primes(kmax)
forpinplist:
print'prime:%d'%p
Page75
cdefprimes(intkmax):
cdefintn,k,i
cdefintp[1000]
result=[]
ifkmax>1000:
kmax=1000
k=0
n=2
whilek<kmax:
i=0
whilei<kandn%p[i]<>0:
i=i+1
ifi==k:
p[k]=n
k=k+1
result.append(n)
n=n+1
returnresult
#CFLAGS=DNDEBUGgO3WallWstrictprototypesfPIC#
I/usr/local/include/python2.3CFLAGS=DNDEBUG
I/usr/local/include/python2.3
all:python_201_pyrex_primes.so
python_201_pyrex_primes.so:python_201_pyrex_primes.o
gccsharedpython_201_pyrex_primes.oopython_201_pyrex_primes.so
python_201_pyrex_primes.o:python_201_pyrex_primes.c
gccc${CFLAGS}python_201_pyrex_primes.copython_201_pyrex_primes.o
python_201_pyrex_primes.c:python_201_pyrex_primes.pyx
pyrexcpython_201_pyrex_primes.pyx
clean:
rmfpython_201_pyrex_primes.sopython_201_pyrex_primes.o
python_201_pyrex_primes.c
Hereistheoutputfromrunningthemakefile:
$makefpython_201_pyrex_makeprimesclean
rmfpython_201_pyrex_primes.sopython_201_pyrex_primes.o\
python_201_pyrex_primes.c
$makefpython_201_pyrex_makeprimes
pyrexcpython_201_pyrex_primes.pyx
gcccDNDEBUGI/usr/local/include/python2.3
python_201_pyrex_primes.copython_201_pyrex_primes.o
gccsharedpython_201_pyrex_primes.oopython_201_pyrex_primes.so
Hereisaninteractiveexampleofitsuse:
$python
Page76
Python2.3b1(#1,Apr252003,20:36:09)
information.
>>>importpython_201_pyrex_primes
>>>dir(python_201_pyrex_primes)
['__builtins__','__doc__','__file__','__name__','showPrimes']
>>>python_201_pyrex_primes.showPrimes(5)
prime:2
prime:3
prime:5
prime:7
prime:11
ThisnextexampleshowshowtousePyrextoimplementanewextensiontype,thatisa
newPythonbuiltintype.Noticethattheclassisdeclaredwiththecdefkeyword,which
tellsPyrextogeneratetheCimplementationofatypeinsteadofaclass.
Hereistheimplementationfile:
#python_201_pyrex_clsprimes.pyx
"""Animplementationofprimeshandlingclass
forademonstrationofPyrex.
"""
cdefclassPrimes:
"""Aclasscontainingfunctionsfor
handlingprimes.
"""
defshowPrimes(self,intkmax):
"""Showarangeofprimes.
Usethemethodprimes()togeneratetheprimes.
"""
plist=self.primes(kmax)
forpinplist:
print'prime:%d'%p
defprimes(self,intkmax):
"""Generatetheprimesintherange0kmax.
"""
cdefintn,k,i
cdefintp[1000]
result=[]
ifkmax>1000:
kmax=1000
k=0
n=2
whilek<kmax:
i=0
whilei<kandn%p[i]<>0:
i=i+1
ifi==k:
p[k]=n
Page77
k=k+1
result.append(n)
n=n+1
returnresult
CFLAGS=DNDEBUGI/usr/local/include/python2.3
all:python_201_pyrex_clsprimes.so
python_201_pyrex_clsprimes.so:python_201_pyrex_clsprimes.o
gccsharedpython_201_pyrex_clsprimes.oo
python_201_pyrex_clsprimes.so
python_201_pyrex_clsprimes.o:python_201_pyrex_clsprimes.c
gccc${CFLAGS}python_201_pyrex_clsprimes.co
python_201_pyrex_clsprimes.o
python_201_pyrex_clsprimes.c:python_201_pyrex_clsprimes.pyx
pyrexcpython_201_pyrex_clsprimes.pyx
clean:
rmfpython_201_pyrex_clsprimes.so
python_201_pyrex_clsprimes.o\
python_201_pyrex_clsprimes.c
Hereisoutputfromrunningthemakefile:
$makefpython_201_pyrex_makeclsprimesclean
rmfpython_201_pyrex_clsprimes.sopython_201_pyrex_clsprimes.o\
python_201_pyrex_clsprimes.c
$makefpython_201_pyrex_makeclsprimes
pyrexcpython_201_pyrex_clsprimes.pyx
gcccDNDEBUGI/usr/local/include/python2.3
python_201_pyrex_clsprimes.copython_201_pyrex_clsprimes.o
gccsharedpython_201_pyrex_clsprimes.oo
python_201_pyrex_clsprimes.so
Andhereisaninteractiveexampleofitsuse:
$python
Python2.3b1(#1,Apr252003,20:36:09)
information.
>>>importpython_201_pyrex_clsprimes
>>>dir(python_201_pyrex_clsprimes)
['Primes','__builtins__','__doc__','__file__','__name__']
>>>primes=python_201_pyrex_clsprimes.Primes()
>>>dir(primes)
['__class__','__delattr__','__doc__','__getattribute__',
'__hash__',
'__init__','__new__','__reduce__','__reduce_ex__','__repr__',
'__setattr__','__str__','primes','showPrimes']
>>>primes.showPrimes(4)
Page78
prime:2
prime:3
prime:5
prime:7
DocumentationAlsonoticethatPyrexpreservesthedocumentationforthemodule,the
class,andthemethodsintheclass.Youcanshowthisdocumentationwithpydoc,as
follows:
$pydocpython_201_pyrex_clsprimes
Or,inPythoninteractivemode,use:
$python
Python2.3b1(#1,Apr252003,20:36:09)
information.
>>>importpython_201_pyrex_clsprimes
>>>help(python_201_pyrex_clsprimes)
2.5.5SWIGvs.Pyrex
ChooseSWIGwhen:
YoualreadyhaveanexistingCorC++implementationofthecodeyouwantto
callfromPython.InthiscaseyouwantSWIGtogeneratethewrappers.Butnote
thatCythonpromisestoenableyoutoquicklywrapandcallfunctions
implementedinC.
YouwanttowritetheimplementationinCorC++byhand.Perhaps,becauseyou
thinkyoucandosoquickly,forexample,orbecauseyoubelievethatyoucan
makeithighlyoptimized.Then,youwanttobeabletogeneratethePython
(extension)wrappersforitquickly.
ChoosePyrexwhen:
YoudonothaveaC/C++implementationandyouwantaneasierwaytowrite
thatCimplementation.WritingPyrexcode,whichisalotlikePython,iseasier
thanwritingCorC++codebyhand).
YoustarttowritetheimplementationinC,thenfindthatitrequireslotsofcallsto
thePythonCAPI,andyouwanttoavoidhavingtolearnhowtodothat.
2.5.6Cython
HereisasimpleexamplethatusesCythontowrapafunctionimplementedinC.
FirsttheCheaderfile:
/*test_c_lib.h*/
intcalculate(intwidth,intheight);
Page79
And,theCimplementationfile:
/*test_c_lib.c*/
#include"test_c_lib.h"
intcalculate(intwidth,intheight)
{
intresult;
result=width*height*3;
returnresult;
}
HereisaCythonfilethatcallsourCfunction:
#test_c.pyx
#DeclaretheexternalCfunction.
cdefexternfrom"test_c_lib.h":
intcalculate(intwidth,intheight)
deftest(w,h):
#CalltheexternalCfunction.
result=calculate(w,h)
print'resultfromcalculate:%d'%result
Wecancompileourcodeusingthisscript(onLinux):
#!/bin/bashx
cythontest_c.pyx
gcccfPICI/usr/local/include/python2.6otest_c.otest_c.c
gcccfPICI/usr/local/include/python2.6otest_c_lib.o
test_c_lib.c
gccsharedfPICI/usr/local/include/python2.6otest_c.so
test_c.otest_c_lib.o
HereisasmallPythonfilethatusesthewrapperthatwewroteinCython:
#run_test_c.py
importtest_c
deftest():
test_c.test(4,5)
test_c.test(12,15)
test()
And,whenwerunit,weseethefollowing:
$pythonrun_test_c.py
resultfromcalculate:60
resultfromcalculate:540
Page80
2.5.7Extensiontypes
ThegoalAnewbuiltindatatypeforPython.
ExistingexamplesObjects/listobject.c,Objects/stringobject.c,Objects/dictobject.c,etc
inthePythonsourcecodedistribution.
InolderversionsofthePythonsourcecodedistribution,atemplatefortheCcodewas
providedinObjects/xxobject.c.Objects/xxobject.cisnolongerincludedinthePython
sourcecodedistribution.However:
Thediscussionandexamplesforcreatingextensiontypeshavebeenexpanded.
See:ExtendingandEmbeddingthePythonInterpreter,2.DefiningNewTypes
http://docs.python.org/extending/newtypes.html.
IntheTools/framerdirectoryofthePythonsourcecodedistributionthereisan
applicationthatwillgenerateaskeletonforanextensiontypefromaspecification
objectwritteninPython.RunTools/framer/example.pytoseeitinaction.
And,youcanusePyrextogenerateanewbuiltintype.Todoso,implementa
Python/PyrexclassanddeclaretheclasswiththePyrexkeywordcdef.Infact,youmay
wanttousePyrextogenerateaminimalextensiontype,andtheneditthatgeneratedcode
toinsertandaddfunctionalitybyhand.SeethePyrexsectionforanexample.
Pyrexalsogoessomewaytowardgivingyouaccessto(existing)Cstructsandfunctions
fromPython.
2.5.8Extensionclasses
ExtensionclassestheeasywaySWIGshadowclasses.
StartwithanimplementationofaC++classanditsheaderfile.
UsethefollowingSWIGflags:
swigc++pythonmymodule.i
MoreinformationisavailablewiththeSWIGdocumentationat:
http://www.swig.org/Doc1.3/Python.html.
ExtensionclassesthePyrexwayAnalternatieistousePyrextocompileaclass
definitionthatdoesnothavethecdefkeyword.UsingcdefontheclasstellsPyrexto
generateanextensiontypeinsteadofaclass.Youwillhavetodeterminewhetheryou
wantanextensionclassoranextensiontype.
2.6Parsing
Pythonisanexcellentlanguagefortextanalysis.
Insomecases,simplysplittinglinesoftextintowordswillbeenough.Inthesecasesuse
string.split().
Page81
Inothercases,regularexpressionsmaybeabletodotheparsingyouneed.Ifso,seethe
sectiononregularexpressionsinthisdocument.
However,insomecases,morecomplexanalysisofinputtextisrequired.Thissection
describessomeofthewaysthatPythoncanhelpyouwiththiscomplexparsingand
analysis.
2.6.1Specialpurposeparsers
ThereareanumberofspecialpurposeparserswhichyouwillfindinthePythonstandard
library:
ConfigParserparserConfigurationfileparser
http://docs.python.org/library/configparser.html
getoptParserforcommandlineoptions
http://docs.python.org/library/getopt.html
optparseMorepowerfulcommandlineoptionparser
http://docs.python.org/library/optparse.html
urlparseParseURLsintocomponents
http://docs.python.org/library/urlparse.html
csvCSV(commaseparatedvalues)FileReadingandWriting
http://docs.python.org/library/csv.html#modulecsv
os.pathCommonpathnamemanipulations
http://docs.python.org/library/os.path.html
XMLparsersandXMLtoolsThereislotsofsupportforparsingandprocessingXML
inPython.Hereareafewplacestolookforsupport:
ThePythonstandardlibraryStructuredMarkupProcessingTools
http://docs.python.org/library/markup.html.
Inparticular,youmaybeinterestedinxml.dom.minidomLightweightDOM
implementationhttp://docs.python.org/library/xml.dom.minidom.html.
ElementTreeYoucanthinkofElementTreeasanenhancedDOM(document
objectmodel).Manyfinditeasiertousethanminidom.ElementTreeisinthe
Pythonstandardlibrary,anddocumentationishere:ElementTreeOverview
http://effbot.org/zone/elementindex.htm.
LxmlmimicstheElementTreeAPI,buthasadditionalcapabilities.Findoutabout
Lxmlatlxmlhttp://codespeak.net/lxml/index.htmlNotethatlxmlalsohas
supportforXPathandXSLT.
Dave'ssupportforPythonandXMLhttp://www.rexx.com/~dkuhlman.
2.6.2Writingarecursivedescentparserbyhand
Forsimplegrammars,thisisnotsohard.
Youwillneedtoimplement:
Page82
Arecognizermethodorfunctionforeachproductionruleinyourgrammar.Each
recognizermethodbeginslookingatthecurrenttoken,thenconsumesasmany
tokensasneededtorecognizeit'sownproductionrule.Itcallstherecognizer
functionsforanynonterminalsonitsrighthandside.
AtokenizerSomethingthatwillenableeachrecognizerfunctiontogettokens,
onebyone.Thereareavarietyofwaystodothis,e.g.(1)afunctionthat
producesalistoftokensfromwhichrecognizerscanpoptokens;(2)agenerator
whosenextmethodreturnsthenexttoken;etc.
Asanexample,we'llimplementarecursivedescentparserwritteninPythonforthe
followinggrammer:
Prog::=Command|CommandProg
Command::=Func_call
Func_call::=Term'('Func_call_list')'
Func_call_list::=Func_call|Func_call','Func_call_list
Term=<word>
Hereisanimplementationofarecursivedescentparserfortheabovegrammar:
#!/usr/bin/envpython
"""
Arecursivedescentparserexample.
Usage:
pythonrparser.py[options]<inputfile>
Options:
h,helpDisplaythishelpmessage.
Example:
pythonrparser.pymyfile.txt
Thegrammar:
Command::=Func_call
Term=<word>
"""
importsys
importstring
importtypes
importgetopt
#
#TousetheIPythoninteractiveshelltoinspectyourrunning
#application,uncommentthefollowinglines:
#
##fromIPython.ShellimportIPShellEmbed
##ipshell=IPShellEmbed((),
##banner='>>>>>>>>IntoIPython>>>>>>>>',
##exit_msg='<<<<<<<<OutofIPython<<<<<<<<')
#
Page83
#Thenaddthefollowinglineatthepointinyourcodewhere
#youwanttoinspectruntimevalues:
#
#ipshell('somemessagetoidentifywhereweare')
#
#Formoreinformationsee:http://ipython.scipy.org/moin/
#
#
#Constants
#
#ASTnodetypes
NoneNodeType=0
ProgNodeType=1
CommandNodeType=2
FuncCallNodeType=3
FuncCallListNodeType=4
TermNodeType=5
#Tokentypes
NoneTokType=0
LParTokType=1
RParTokType=2
WordTokType=3
CommaTokType=4
EOFTokType=5
#Dictionarytomapnodetypevaluestonodetypenames
NodeTypeDict={
NoneNodeType:'NoneNodeType',
ProgNodeType:'ProgNodeType',
CommandNodeType:'CommandNodeType',
FuncCallNodeType:'FuncCallNodeType',
FuncCallListNodeType:'FuncCallListNodeType',
TermNodeType:'TermNodeType',
}
#
#RepresentationofanodeintheAST(abstractsyntaxtree).
#
classASTNode:
def__init__(self,nodeType,*args):
self.nodeType=nodeType
self.children=[]
foriteminargs:
self.children.append(item)
defshow(self,level):
self.showLevel(level)
print'NodeType%s'%NodeTypeDict[self.nodeType]
level+=1
ifisinstance(child,ASTNode):
child.show(level)
eliftype(child)==types.ListType:
foriteminchild:
Page84
item.show(level)
else:
print'Child:',child
defshowLevel(self,level):
foridxinrange(level):
print'',
#
#Therecursivedescentparserclass.
#Containsthe"recognizer"methods,whichimplementthegrammar
#rules(above),onerecognizermethodforeachproductionrule.
#
classProgParser:
def__init__(self):
pass
defparseFile(self,infileName):
self.infileName=infileName
self.tokens=None
self.tokenType=NoneTokType
self.token=''
self.lineNo=1
self.infile=file(self.infileName,'r')
self.tokens=genTokens(self.infile)
try:
self.tokenType,self.token,self.lineNo=
self.tokens.next()
exceptStopIteration:
raiseRuntimeError,'Emptyfile'
result=self.prog_reco()
self.infile.close()
self.infile=None
returnresult
defparseStream(self,instream):
self.tokens=genTokens(instream,'<instream>')
try:
self.tokens.next()
returnresult
defprog_reco(self):
commandList=[]
while1:
result=self.command_reco()
ifnotresult:
break
commandList.append(result)
returnASTNode(ProgNodeType,commandList)
defcommand_reco(self):
ifself.tokenType==EOFTokType:
Page85
returnNone
result=self.func_call_reco()
returnASTNode(CommandNodeType,result)
deffunc_call_reco(self):
ifself.tokenType==WordTokType:
term=ASTNode(TermNodeType,self.token)
self.tokens.next()
ifself.tokenType==LParTokType:
self.tokens.next()
result=self.func_call_list_reco()
ifresult:
ifself.tokenType==RParTokType:
self.tokenType,self.token,self.lineNo=\
self.tokens.next()
returnASTNode(FuncCallNodeType,term,
result)
else:
raiseParseError(self.lineNo,'missingright
paren')
else:
raiseParseError(self.lineNo,'badfunccall
list')
else:
raiseParseError(self.lineNo,'missingleftparen')
else:
returnNone
deffunc_call_list_reco(self):
terms=[]
while1:
ifnotresult:
break
terms.append(result)
ifself.tokenType!=CommaTokType:
break
self.tokens.next()
returnASTNode(FuncCallListNodeType,terms)
#
#Theparseerrorexceptionclass.
#
classParseError(Exception):
def__init__(self,lineNo,msg):
RuntimeError.__init__(self,msg)
self.lineNo=lineNo
self.msg=msg
defgetLineNo(self):
returnself.lineNo
defgetMsg(self):
returnself.msg
Page86
defis_word(token):
forletterintoken:
ifletternotinstring.ascii_letters:
returnNone
return1
#
#Generatethetokens.
#Usage:
#gen=genTokens(infile)
#tokType,tok,lineNo=gen.next()
#...
defgenTokens(infile):
lineNo=0
while1:
lineNo+=1
try:
line=infile.next()
except:
yield(EOFTokType,None,lineNo)
toks=line.split()
fortokintoks:
ifis_word(tok):
tokType=WordTokType
eliftok=='(':
tokType=LParTokType
eliftok==')':
tokType=RParTokType
eliftok==',':
tokType=CommaTokType
yield(tokType,tok,lineNo)
deftest(infileName):
parser=ProgParser()
#ipshell('(test)#1\nCtrlDtoexit')
result=None
try:
result=parser.parseFile(infileName)
exceptParseError,exp:
sys.stderr.write('ParseError:(%d)%s\n'%\
(exp.getLineNo(),exp.getMsg()))
ifresult:
result.show(0)
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
relink=1
foropt,valinopts:
Page87
ifoptin('h','help'):
usage()
iflen(args)!=1:
usage()
inputfile=args[0]
test(inputfile)
#importpdb;pdb.set_trace()
main()
Commentsandexplanation:
ThetokenizerisaPythongenerator.ItreturnsaPythongeneratorthatcan
produce"(tokType,tok,lineNo)"tuples.Ourtokenizerissosimplemindedthat
wehavetoseparateallofourtokenswithwhitespace.(Alittlelater,we'llseehow
tousePlextoovercomethislimitation.)
Theparserclass(ProgParser)containstherecognizermethodsthatimplementthe
productionrules.Eachofthesemethodsrecognizesasyntacticconstructdefined
byarule.Inourexample,thesemethodshavenamesthatendwith"_reco".
Wecouldhave,alternatively,implementedourrecognizersasglobalfunctions,
insteadofasmethodsinaclass.However,usingaclassgivesusaplaceto"hang"
thevariablesthatareneededacrossmethodsandsavesusfromhavingtouse
("evil")globalvariables.
Arecognizermethodrecognizesterminals(syntacticelementsontherighthand
sideofthegrammarruleforwhichthereisnogrammarrule)by(1)checkingthe
tokentypeandthetokenvalue,andthen(2)callingthetokenizertogetthenext
token(becauseithasconsumedatoken).
Arecognizermethodchecksforandprocessesanonterminal(syntacticelements
ontherighthandsideforwhichthereisagrammarrule)bycallingtherecognizer
methodthatimplementsthatnonterminal.
Ifarecognizermethodfindsasyntaxerror,itraisesanexceptionofclass
ParserError.
SinceourexamplerecursivedescentparsercreatesanAST(anabstractsyntax
tree),wheneverarecognizermethodsuccessfullyrecognizesasyntacticconstruct,
itcreatesaninstanceofclassASTNodetorepresentitandreturnsthatinstanceto
itscaller.TheinstanceofASTNodehasanodetypeandcontainschildnodes
whichwereconstructedbyrecognizermethodscalledbythisone(i.e.that
representnonterminalsontherighthandsideofagrammarrule).
Eachtimearecognizermethod"consumesatoken",itcallsthetokenizertoget
thenexttoken(andtokentypeandlinenumber).
Thetokenizerreturnsatokentypeinadditiontothetokenvalue.Italsoreturnsa
linenumberforerrorreporting.
ThesyntaxtreeisconstructedfrominstancesofclassASTNode.
TheASTNodeclasshasashowmethod,whichwalkstheASTandproduces
output.Youcanimaginethatasimilarmethodcoulddocodegeneration.And,
Page88
youshouldconsiderthepossibilityofwritinganalogoustreewalkmethodsthat
performtaskssuchasoptimization,annotationoftheAST,etc.
And,hereisasampleofthedatawecanapplythisparserto:
aaa()
bbb(ccc())
ddd(eee(),fff(ggg(),hhh(),iii()))
And,ifweruntheparseronthethisinputdata,wesee:
$pythonworkbook045.pyworkbook045.data
NodeTypeProgNodeType
NodeTypeCommandNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:aaa
NodeTypeFuncCallListNodeType
Child:bbb
Child:ccc
Child:ddd
Child:eee
Child:fff
Child:ggg
Child:hhh
Child:iii
Page89
2.6.3Creatingalexer/tokenizerwithPlex
LexicalanalysisThetokenizerinourrecursivedescentparserexamplewas(for
demonstrationpurposes)overlysimple.Youcanalwayswritemorecomplextokenizers
byhand.However,formorecomplex(andreal)tokenizers,youmaywanttouseatoolto
buildyourtokenizer.
Inthissectionwe'lldescribePlexanduseittoproduceatokenizerforourrecursive
descentparser.
YoucanobtainPlexathttp://www.cosc.canterbury.ac.nz/~greg/python/Plex/.
Inordertouseit,youmaywanttoaddPlex1.1.4/PlextoyourPYTHONPATH.
HereisasimpleexamplefromthePlextutorial:
"""
SamplePlexlexer
Usage:
pythonplex_example.pyinputfile
"""
importsys
importPlex
defcount_lines(scanner,text):
scanner.line_count+=1
print''*60
letter=Plex.Range("AZaz")
digit=Plex.Range("09")
name=letter+Plex.Rep(letter|digit)
number=Plex.Rep1(digit)
space=Plex.Any("\t")
endline=Plex.Str('\n')
#comment=Plex.Str('"')+Plex.Rep(Plex.AnyBut('"'))+
Plex.Str('"')
resword=Plex.Str("if","then","else","end")
lexicon=Plex.Lexicon([
(endline,count_lines),
(resword,'keyword'),
(name,'ident'),
(number,'int'),
(Plex.Any("+*/=<>"),'operator'),
(space,Plex.IGNORE),
#(comment,'comment'),
(Plex.Str('('),'lpar'),
(Plex.Str(')'),'rpar'),
#commentssurroundedby(*and*)
(Plex.Str("(*"),Plex.Begin('comment')),
Plex.State('comment',[
Page90
(Plex.Str("*)"),Plex.Begin('')),
(Plex.AnyChar,Plex.IGNORE),
]),
])
infile=open(infileName,"r")
scanner=Plex.Scanner(lexicon,infile,infileName)
scanner.line_count=0
whileTrue:
token=scanner.read()
iftoken[0]isNone:
break
position=scanner.position()
posstr=('(%d,%d)'%(position[1],
position[2],)).ljust(10)
tokstr='"%s"'%token[1]
tokstr=tokstr.ljust(20)
print'%stok:%stokType:%s'%(posstr,tokstr,token[0],)
print'line_count:%d'%scanner.line_count
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)!=1:
usage()
infileName=args[0]
test(infileName)
main()
Hereisabitofdataonwhichwecanusetheabovelexer:
mass=(height*(*somecomment*)width*depth)/density
totalmass=totalmass+mass
And,whenweapplytheabovetestprogramtothisdata,hereiswhatwesee:
$pythonplex_example.pyplex_example.data
(1,0)tok:"mass"tokType:ident
(1,5)tok:"="tokType:operator
(1,7)tok:"("tokType:lpar
(1,8)tok:"height"tokType:ident
(1,15)tok:"*"tokType:operator
(1,36)tok:"width"tokType:ident
(1,42)tok:"*"tokType:operator
(1,44)tok:"depth"tokType:ident
(1,49)tok:")"tokType:rpar
(1,51)tok:"/"tokType:operator
(1,53)tok:"density"tokType:ident
(2,0)tok:"totalmass"tokType:ident
Page91
(2,10)tok:"="tokType:operator
(2,12)tok:"totalmass"tokType:ident
(2,22)tok:"+"tokType:operator
(2,24)tok:"mass"tokType:ident
line_count:2
Createalexiconfromscanningpatterns.
SeethePlextutorialandreference(andbelow)formoreinformationonhowto
constructthepatternsthatmatchvarioustokens.
Createascannerwithalexicon,aninputfile,andaninputfilename.
Thecall"scanner.read()"getsthenexttoken.Itreturnsatuplecontaining(1)the
tokenvalueand(2)thetokentype.
Thecall"scanner.position()"getsthepositionofthecurrenttoken.Itreturnsa
tuplecontaining(1)theinputfilename,(2)thelinenumber,and(3)thecolumn
number.
Wecanexecuteamethodwhenagiventokenisfoundbyspecifyingthefunction
asthetokenaction.Inourexample,thefunctioniscount_lines.Maintainingaline
countisactuallyunneeded,sincethepositiongivesusthisinformation.However,
noticehowweareabletomaintainavalue(inourcaseline_count)asan
attributeofthescanner.
And,herearesomecommentsonconstructingthepatternsusedinalexicon:
Plex.Rangeconstructsapatternthatmatchesanycharacterintherange.
Plex.Repconstructsapatternthatmatchesasequenceofzeroormoreitems.
Plex.Rep1constructsapatternthatmatchesasequenceofoneormoreitems.
pat1+pat2constructsapatternthatmatchesasequencecontainingpat1
followedbypat2.
pat1|pat2constructsapatternthatmatcheseitherpat1orpat2.
Plex.Anyconstructsapatternthatmatchesanyonecharacterinitsargument.
Nowlet'srevisitourrecursivedescentparser,thistimewithatokenizerbuiltwithPlex.
Thetokenizeristrivial,butwillserveasanexampleofhowtohookitintoaparser:
"""
ArecursivedescentparserexampleusingPlex.
ThisexampleusesPlextoimplementatokenizer.
Usage:
pythonpython_201_rparser_plex.py[options]<inputfile>
Options:
Example:
pythonpython_201_rparser_plex.pymyfile.txt
Thegrammar:
Page92
Command::=Func_call
Term=<word>
"""
importsys,string,types
importgetopt
importPlex
##fromIPython.ShellimportIPShellEmbed
##ipshell=IPShellEmbed((),
##banner='>>>>>>>>IntoIPython>>>>>>>>',
##exit_msg='<<<<<<<<OutofIPython<<<<<<<<')
#
#Constants
#
#ASTnodetypes
NoneNodeType=0
ProgNodeType=1
CommandNodeType=2
FuncCallNodeType=3
TermNodeType=5
#Tokentypes
NoneTokType=0
LParTokType=1
RParTokType=2
WordTokType=3
CommaTokType=4
EOFTokType=5
NodeTypeDict={
}
#
#
classASTNode:
self.children=[]
foriteminargs:
Page93
print'NodeType%s'%NodeTypeDict[self.nodeType]
level+=1
child.show(level)
foriteminchild:
item.show(level)
else:
print'Child:',child
print'',
#
#Therecursivedescentparserclass.
#Containsthe"recognizer"methods,whichimplementthegrammar
#rules(above),onerecognizermethodforeachproductionrule.
#
classProgParser:
def__init__(self):
self.tokens=None
self.token=''
self.lineNo=1
self.infile=None
self.tokens=None
defparseFile(self,infileName):
self.tokens=None
self.token=''
self.lineNo=1
self.infile=file(infileName,'r')
self.tokens=genTokens(self.infile,infileName)
try:
self.tokens.next()
self.infile.close()
self.infile=None
returnresult
defparseStream(self,instream):
self.tokens=None
self.token=''
self.lineNo=1
self.tokens=genTokens(self.instream,'<stream>')
try:
Page94
self.tokens.next()
raiseRuntimeError,'Emptystream'
self.infile.close()
self.infile=None
returnresult
defprog_reco(self):
commandList=[]
while1:
result=self.command_reco()
ifnotresult:
break
commandList.append(result)
returnASTNode(ProgNodeType,commandList)
defcommand_reco(self):
ifself.tokenType==EOFTokType:
returnNone
returnASTNode(CommandNodeType,result)
deffunc_call_reco(self):
ifself.tokenType==WordTokType:
term=ASTNode(TermNodeType,self.token)
self.tokens.next()
ifself.tokenType==LParTokType:
self.tokens.next()
result=self.func_call_list_reco()
ifresult:
ifself.tokenType==RParTokType:
self.tokenType,self.token,self.lineNo=\
self.tokens.next()
returnASTNode(FuncCallNodeType,term,
result)
else:
raiseParseError(self.lineNo,'missingright
paren')
else:
raiseParseError(self.lineNo,'badfunccall
list')
else:
raiseParseError(self.lineNo,'missingleftparen')
else:
returnNone
deffunc_call_list_reco(self):
terms=[]
while1:
ifnotresult:
break
Page95
terms.append(result)
ifself.tokenType!=CommaTokType:
break
self.tokens.next()
returnASTNode(FuncCallListNodeType,terms)
#
#Theparseerrorexceptionclass.
#
classParseError(Exception):
def__init__(self,lineNo,msg):
RuntimeError.__init__(self,msg)
self.lineNo=lineNo
self.msg=msg
defgetLineNo(self):
returnself.lineNo
defgetMsg(self):
returnself.msg
#
#Generatethetokens.
#Usageexample
#gen=genTokens(infile)
#tokType,tok,lineNo=gen.next()
#...
defgenTokens(infile,infileName):
letter=Plex.Range("AZaz")
digit=Plex.Range("09")
name=letter+Plex.Rep(letter|digit)
lpar=Plex.Str('(')
rpar=Plex.Str(')')
comma=Plex.Str(',')
comment=Plex.Str("#")+Plex.Rep(Plex.AnyBut("\n"))
space=Plex.Any("\t\n")
lexicon=Plex.Lexicon([
(name,'word'),
(lpar,'lpar'),
(rpar,'rpar'),
(comma,'comma'),
(comment,Plex.IGNORE),
(space,Plex.IGNORE),
])
scanner=Plex.Scanner(lexicon,infile,infileName)
while1:
tokenType,token=scanner.read()
name,lineNo,columnNo=scanner.position()
iftokenType==None:
tokType=EOFTokType
token=None
eliftokenType=='word':
tokType=WordTokType
eliftokenType=='lpar':
tokType=LParTokType
eliftokenType=='rpar':
tokType=RParTokType
Page96
eliftokenType=='comma':
tokType=CommaTokType
else:
tokType=NoneTokType
tok=token
yield(tokType,tok,lineNo)
parser=ProgParser()
#ipshell('(test)#1\nCtrlDtoexit')
result=None
try:
result=parser.parseFile(infileName)
exceptParseError,exp:
sys.stderr.write('ParseError:(%d)%s\n'%\
(exp.getLineNo(),exp.getMsg()))
ifresult:
result.show(0)
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
except:
usage()
foropt,valinopts:
usage()
iflen(args)!=1:
usage()
infileName=args[0]
test(infileName)
main()
And,hereisasampleofthedatawecanapplythisparserto:
#TestforrecursivedescentparserandPlex.
#Command#1
aaa()
#Command#2
bbb(ccc())#Anendoflinecomment.
#Command#3
#Endoftest
And,whenwerunourparser,itproducesthefollowing:
$pythonplex_recusive.pyplex_recusive.data
NodeTypeProgNodeType
Page97
Child:aaa
Child:bbb
Child:ccc
Child:ddd
Child:eee
Child:fff
Child:ggg
Child:hhh
Child:iii
Comments:
Wecannowputcommentsinourinput,andtheywillbeignored.Comments
beginwitha"#"andcontinuetotheendofline.Seethedefinitionofcommentin
functiongenTokens.
Thistokenizerdoesnotrequireustoseparatetokenswithwhitespaceasdidthe
simpletokenizerintheearlierversionofourrecursivedescentparser.
Thechangeswemadeovertheearlierversionwereto:
1. ImportPlex.
2. ReplacethedefinitionofthetokenizerfunctiongenTokens.
3. ChangethecalltogenTokenssothatthecallpassesinthefilename,whichis
neededtocreatethescanner.
OurnewversionofgenTokensdoesthefollowing:
Page98
1.
2.
3.
4.
Createpatternsforscanning.
Createalexicon(aninstanceofPlex.Lexicon),whichusesthepatterns.
Createascanner(aninstanceofPlex.Scanner),whichusesthelexicon.
Executealoopthatreadstokens(fromthescanner)and"yields"eachone.
2.6.4Asurveyofexistingtools
Forcomplexparsingtasks,youmaywanttoconsiderthefollowingtools:
kwParsingAparsergeneratorinPython
http://gadfly.sourceforge.net/kwParsing.html
PLYPythonLexYacchttp://systems.cs.uchicago.edu/ply/
PyLRFastLRparsinginpython
http://starship.python.net/crew/scott/PyLR.html
YappsTheYappsParserGeneratorSystem
http://theory.stanford.edu/~amitp/Yapps/
And,forlexicalanalysis,youmayalsowanttolookhere:
UsingRegularExpressionsforLexicalAnalysishttp://effbot.org/zone/xml
scanner.htm
Plexhttp://www.cosc.canterbury.ac.nz/~greg/python/Plex/.
Inthesectionsbelow,wegiveexamplesandnotesabouttheuseofPLYandpyparsing.
2.6.5CreatingaparserwithPLY
InthissectionwewillshowhowtoimplementourparserexamplewithPLY.
FirstdownloadPLY.Itisavailablehere:PLY(PythonLexYacc)
http://www.dabeaz.com/ply/
ThenaddthePLYdirectorytoyourPYTHONPATH.
LearnhowtoconstructlexersandparserswithPLYbyreadingdoc/ply.htmlinthe
distributionofPLYandbylookingattheexamplesinthedistribution.
Forthoseofyouwhowantamorecomplexexample,seeAPythonParserforthe
RELAXNGCompactSyntax,whichisimplementedwithPLY.
Now,hereisourexampleparser.Commentsandexplanationsarebelow:
"""
Aparserexample.
ThisexampleusesPLYtoimplementalexerandparser.
Thegrammar:
Prog::=Command*
Command::=Func_call
Page99
Func_call_list::=Func_call*
Term=<word>
Hereisasample"program"touseasinput:
#Command#1
aaa()
#Command#2
#Command#3
#Endoftest
"""
importsys
importtypes
importgetopt
importply.lexaslex
importply.yaccasyacc
#
#Globals
#
startlinepos=0
#
#Constants
#
#ASTnodetypes
NoneNodeType=0
ProgNodeType=1
CommandNodeType=2
CommandListNodeType=3
FuncCallNodeType=4
TermNodeType=6
NodeTypeDict={
CommandListNodeType:'CommandListNodeType',
}
#
#
classASTNode:
Page100
self.children=[]
foriteminargs:
defappend(self,item):
print'NodeType:%s'%NodeTypeDict[self.nodeType]
level+=1
child.show(level)
foriteminchild:
item.show(level)
else:
print'Value:',child
print'',
#
#Exceptionclasses
#
classLexerError(Exception):
def__init__(self,msg,lineno,columnno):
self.msg=msg
self.lineno=lineno
self.columnno=columnno
defshow(self):
sys.stderr.write('Lexererror(%d,%d)%s\n'%\
(self.lineno,self.columnno,self.msg))
classParserError(Exception):
def__init__(self,msg,lineno,columnno):
self.msg=msg
self.lineno=lineno
self.columnno=columnno
defshow(self):
sys.stderr.write('Parsererror(%d,%d)%s\n'%\
(self.lineno,self.columnno,self.msg))
#
#Lexerspecification
#
tokens=(
'NAME',
'LPAR','RPAR',
'COMMA',
)
#Tokens
Page101
t_LPAR=r'$'
t_RPAR=r'$'
t_COMMA=r'\,'
t_NAME=r'[azAZ_][azAZ09_]*'
#Ignorewhitespace
t_ignore='\t'
#Ignorecomments('#'toendofline)
deft_COMMENT(t):
r'\#[^\n]*'
pass
deft_newline(t):
r'\n+'
globalstartlinepos
startlinepos=t.lexer.lexpos1
t.lineno+=t.value.count("\n")
deft_error(t):
globalstartlinepos
msg="Illegalcharacter'%s'"%(t.value[0])
columnno=t.lexer.lexposstartlinepos
raiseLexerError(msg,t.lineno,columnno)
#
#Parserspecification
#
defp_prog(t):
'prog:command_list'
t[0]=ASTNode(ProgNodeType,t[1])
defp_command_list_1(t):
'command_list:command'
t[0]=ASTNode(CommandListNodeType,t[1])
defp_command_list_2(t):
'command_list:command_listcommand'
t[1].append(t[2])
t[0]=t[1]
defp_command(t):
'command:func_call'
t[0]=ASTNode(CommandNodeType,t[1])
defp_func_call_1(t):
'func_call:termLPARRPAR'
t[0]=ASTNode(FuncCallNodeType,t[1])
defp_func_call_2(t):
'func_call:termLPARfunc_call_listRPAR'
t[0]=ASTNode(FuncCallNodeType,t[1],t[3])
defp_func_call_list_1(t):
'func_call_list:func_call'
t[0]=ASTNode(FuncCallListNodeType,t[1])
Page102
defp_func_call_list_2(t):
'func_call_list:func_call_listCOMMAfunc_call'
t[1].append(t[3])
t[0]=t[1]
defp_term(t):
'term:NAME'
t[0]=ASTNode(TermNodeType,t[1])
defp_error(t):
globalstartlinepos
msg="Syntaxerrorat'%s'"%t.value
columnno=t.lexer.lexposstartlinepos
raiseParserError(msg,t.lineno,columnno)
#
#ParsetheinputanddisplaytheAST(abstractsyntaxtree)
#
defparse(infileName):
startlinepos=0
#Buildthelexer
lex.lex(debug=1)
#Buildtheparser
yacc.yacc()
#Readtheinput
infile=file(infileName,'r')
content=infile.read()
infile.close()
try:
#Dotheparse
result=yacc.parse(content)
#DisplaytheAST
result.show(0)
exceptLexerError,exp:
exp.show()
exceptParserError,exp:
exp.show()
USAGE_TEXT=__doc__
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
except:
usage()
relink=1
foropt,valinopts:
usage()
iflen(args)!=1:
Page103
usage()
infileName=args[0]
parse(infileName)
main()
Applyingthisparsertothefollowinginput:
#Command#1
aaa()
#Command#2
#Command#3
#Endoftest
producesthefollowingoutput:
NodeType:ProgNodeType
NodeType:CommandListNodeType
NodeType:CommandNodeType
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:aaa
Value:bbb
NodeType:FuncCallListNodeType
Value:ccc
Value:ddd
Value:eee
Value:fff
Value:ggg
Value:hhh
Value:iii
Page104
CreatingthesyntaxtreeBasically,eachrule(1)recognizesanonterminal,(2)
createsanode(possiblyusingthevaluesfromtherighthandsideoftherule),and
(3)returnsthenodebysettingthevalueoft[0].Adeviationfromthisisthe
processingofsequences,discussedbelow.
Sequencesp_command_list_1andp_command_list_1showhowtohandle
sequencesofitems.Inthiscase:
p_command_list_1recognizesacommandandcreatesaninstanceof
ASTNodewithtypeCommandListNodeTypeandaddsthecommandtoitasa
child,and
p_command_list_2recognizesanadditionalcommandandaddsit(asachild)
totheinstanceofASTNodethatrepresentsthelist.
DistinguishingbetweendifferentformsofthesameruleInordertoprocess
alternativestothesameproductionruledifferently,weusedifferentfunctions
withdifferentimplementations.Forexample,weuse:
p_func_call_1torecognizeandprocess"func_call:termLPARRPAR"(a
functioncallwithoutarguments),and
p_func_call_2torecognizeandprocess"func_call:termLPARfunc_call_list
RPAR"(afunctioncallwitharguments).
ReportingerrorsOurparserreportsthefirsterrorandquits.We'vedonethisby
raisinganexceptionwhenwefindanerror.Weimplementtwoexceptionclasses:
LexerErrorandParserError.Implementingmorethanoneexceptionclassenables
ustodistinguishbetweendifferentclassesoferrors(notethemultipleexcept:
clausesonthetry:statementinfunctionparse).And,weuseaninstanceofthe
exceptionclassasacontainerinorderto"bubbleup"informationabouttheerror
(e.g.amessage,alinenumber,andacolumnnumber).
2.6.6Creatingaparserwithpyparsing
pyparsingisarelativelynewparsingpackageforPython.Itwasimplementedandis
supportedbyPaulMcGuireanditshowspromise.Itappearsespeciallyeasytouseand
seemsespeciallyappropriateinparticularforquickparsingtasks,althoughithasfeatures
thatmakesomecomplexparsingtaskseasy.ItfollowsaverynaturalPythonstylefor
constructingparsers.
Gooddocumentationcomeswiththepyparsingdistribution.Seefile
HowToUseParsing.html.So,Iwon'ttrytorepeatthathere.Whatfollowsisanattemptto
provideseveralquickexamplestohelpyousolvesimpleparsingtasksasquicklyas
possible.
Youwillalsowanttolookatthesamplesintheexamplesdirectory,whicharevery
helpful.Myexamplesbelowarefairlysimple.Youcanseemoreoftheabilityof
pyparsingtohandlecomplextasksintheexamples.
Page105
WheretogetitYoucanfindpyparsingat:PyparsingWikiHome
http://pyparsing.wikispaces.com/
HowtoinstallitPutthepyparsingmodulesomewhereonyourPYTHONPATH.
Andnow,hereareafewexamples.
2.6.6.1Parsingcommadelimitedlines
Note:Thisexampleisfordemonstrationpurposesonly.Ifyoureallytoneedtoparse
commadelimitedfields,youcanprobablydosomuchmoreeasilywiththeCSV(comma
separatedvalues)moduleinthePythonstandardlibrary.
Hereisasimplegrammarforlinescontainingfieldsseparatedbycommas:
importsys
frompyparsingimportalphanums,ZeroOrMore,Word
fieldDef=Word(alphanums)
lineDef=fieldDef+ZeroOrMore(","+fieldDef)
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print'usage:pythonpyparsing_test1.py<datafile.txt>'
sys.exit(1)
infilename=sys.argv[1]
forlineininfile:
fields=lineDef.parseString(line)
printfields
test()
Hereissomesampledata:
abcd,defg
11111,22222,33333
And,whenwerunourparseronthisdatafile,hereiswhatwesee:
$pythoncomma_parser.pysample1.data
['abcd',',','defg']
['11111',',','22222',',','33333']
NotehowthegrammarisconstructedfromnormalPythoncallstofunctionand
object/classconstructors.I'veconstructedtheparserinlinebecausemyexample
issimple,butconstructingtheparserinafunctionorevenamodulemightmake
senseformorecomplexgrammars.pyparsingmakesiteasytousethesethese
differentstyles.
Use"+"tospecifyasequence.Inourexample,alineDefisafieldDef
Page106
followedby....
UseZeroOrMoretospecifyrepetition.Inourexample,alineDefisa
fieldDeffollowedbyzeroormoreoccurancesofcommaandfieldDef.
ThereisalsoOneOrMorewhenyouwanttorequireatleastoneoccurance.
Parsingcommadelimitedtexthappenssofrequentlythatpyparsingprovidesa
shortcut.Replace:
lineDef=fieldDef+ZeroOrMore(","+fieldDef)
with:
lineDef=delimitedList(fieldDef)
AndnotethatdelimitedListtakesanoptionalargumentdelimusedtospecify
thedelimiter.Thedefaultisacomma.
2.6.6.2Parsingfunctors
Thisexampleparsesexpressionsoftheformfunc(arg1,arg2,arg3):
frompyparsingimportWord,alphas,alphanums,nums,ZeroOrMore,
Literal
lparen=Literal("(")
rparen=Literal(")")
identifier=Word(alphas,alphanums+"_")
integer=Word(nums)
functor=identifier
arg=identifier|integer
args=arg+ZeroOrMore(","+arg)
expression=functor+lparen+args+rparen
deftest():
content=raw_input("Enteranexpression:")
parsedContent=expression.parseString(content)
printparsedContent
test()
Explanation:
UseLiteraltospecifyafixedstringthatistobematchedexactly.Inourexample,
alparenisa(.
Wordtakesanoptionalsecondargument.Withasingle(string)argument,it
matchesanycontiguouswordmadeupofcharactersinthestring.Withtwo
(string)argumentsitmatchesawordwhosefirstcharacterisinthefirststringand
whoseremainingcharactersareinthesecondstring.So,ourdefinitionof
identifiermatchesawordwhosefirstcharacterisanalphaandwhoseremaining
charactersarealphanumericsorunderscore.Asanotherexample,youcanthink
ofWord("0123456789")asanalogoustoaregexpcontainingthepattern"[09]+".
Useaverticalbarforalternation.Inourexample,anargcanbeeitheranidentifier
oraninteger.
Page107
2.6.6.3Parsingnames,phonenumbers,etc.
Thisexampleparsesexpressionshavingthefollowingform:
Inputformat:
[name][phone][city,statezip]
Last,first1112223333city,ca99999
Hereistheparser:
importsys
frompyparsingimportalphas,nums,ZeroOrMore,Word,Group,
Suppress,Combine
lastname=Word(alphas)
firstname=Word(alphas)
city=Group(Word(alphas)+ZeroOrMore(Word(alphas)))
state=Word(alphas,exact=2)
zip=Word(nums,exact=5)
name=Group(lastname+Suppress(",")+firstname)
phone=Combine(Word(nums,exact=3)+""+Word(nums,exact=3)+""
+Word(nums,exact=4))
location=Group(city+Suppress(",")+state+zip)
record=name+phone+location
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print'usage:pythonpyparsing_test3.py<datafile.txt>'
sys.exit(1)
infilename=sys.argv[1]
forlineininfile:
line=line.strip()
iflineandline[0]!="#":
fields=record.parseString(line)
printfields
test()
And,hereissomesampleinput:
Jabberer,Jerry1112223333Bakersfield,CA95111
Kackler,Kerry1112223334Fresno,CA95112
Louderdale,Larry1112223335LosAngeles,CA94001
Hereisoutputfromparsingtheaboveinput:
[['Jabberer','Jerry'],'1112223333',[['Bakersfield'],'CA',
'95111']]
[['Kackler','Kerry'],'1112223334',[['Fresno'],'CA','95112']]
[['Louderdale','Larry'],'1112223335',[['Los','Angeles'],'CA',
'94001']]
Page108
Comments:
Weusethelen=nargumenttotheWordconstructortoresticttheparserto
acceptingaspecificnumberofcharacters,forexampleinthezipcodeandphone
number.Wordalsoacceptsmin=n''and``max=ntoenableyoutorestrict
thelengthofawordtowithinarange.
WeuseGrouptogrouptheparsedresultsintosublists,forexampleinthe
definitionofcityandname.Groupenablesustoorganizetheparseresultsinto
simpleparsetrees.
WeuseCombinetojoinparsedresultsbackintoasinglestring.Forexample,in
thephonenumber,wecanrequiredashesandyetjointheresultsbackintoa
singlestring.
WeuseSuppresstoremoveunneededsubelementsfromparsedresults.For
example,wedonotneedthecommabetweenlastandfirstname.
2.6.6.4Amorecomplexexample
Thisexample(thankstoPaulMcGuire)parsesamorecomplexstructureandproducesa
dictionary.
Hereisthecode:
frompyparsingimportLiteral,Word,Group,Dict,ZeroOrMore,alphas,
nums,\
delimitedList
importpprint
testData="""
++++++++++
||A1|B1|C1|D1|A2|B2|C2|D2|
+=======+======+======+======+======+======+======+======+======+
|min|7|43|7|15|82|98|1|37|
|max|11|52|10|17|85|112|4|39|
|ave|9|47|8|16|84|106|3|38|
|sdev|1|3|1|1|1|3|1|1|
++++++++++
"""
#Definegrammarfordatatable
heading=(Literal(
"++++++++++")
+
"||A1|B1|C1|D1|A2|B2|C2|D2|"+
"+=======+======+======+======+======+======+======+======+======+").
suppress()
vert=Literal("|").suppress()
number=Word(nums)
rowData=Group(vert+Word(alphas)+vert+
delimitedList(number,"|")+
vert)
trailing=Literal(
Page109
"+++++++++
+").suppress()
datatable=heading+Dict(ZeroOrMore(rowData))+trailing
defmain():
#Nowparsedataandprintresults
data=datatable.parseString(testData)
print"data:",data
print"data.asList():",
pprint.pprint(data.asList())
print"datakeys:",data.keys()
print"data['min']:",data['min']
print"data.max:",data.max
main()
Whenwerunthis,itproducesthefollowing:
data:[['min','7','43','7','15','82','98','1','37'],
['max','11','52','10','17','85','112','4','39'],
['ave','9','47','8','16','84','106','3','38'],
['sdev','1','3','1','1','1','3','1','1']]
data.asList():[['min','7','43','7','15','82','98','1','37'],
['max','11','52','10','17','85','112','4','39'],
['ave','9','47','8','16','84','106','3','38'],
['sdev','1','3','1','1','1','3','1','1']]
datakeys:['ave','min','sdev','max']
data['min']:['7','43','7','15','82','98','1','37']
data.max:['11','52','10','17','85','112','4','39']
Notes:
NotetheuseofDicttocreateadictionary.Theprintstatementsshowhowtoget
attheitemsinthedictionary.
NotehowwecanalsogettheparseresultsasalistbyusingmethodasList.
Again,weusesuppresstoremoveunneededitemsfromtheparseresults.
2.7GUIApplications
2.7.1Introduction
ThissectionwillhelpyoutoputaGUI(graphicaluserinterface)inyourPython
program.
WewilluseaparticularGUIlibrary:PyGTK.We'vechosenthisbecauseitisreasonably
lightweightandourgoalistoembedlightweightGUIinterfacesinan(possibly)
existingapplication.
ForsimplerGUIneeds,considerEasyGUI,whichisalsodescribedbelow.
FormoreheavyweightGUIneeds(forexample,completeGUIapplications),youmay
Page110
wanttoexploreWxPython.SeetheWxPythonhomepageat:http://www.wxpython.org/
2.7.2PyGtk
InformationaboutPyGTKishere:ThePyGTKhomepagehttp://www.pygtk.org//.
2.7.2.1Asimplemessagedialogbox
InthissectionweexplainhowtopopupasimpledialogboxfromyourPython
application.
Todothis,dothefollowing:
1. ImportgtkintoyourPythonmodule.
2. Definethedialoganditsbehavior.
3. Createaninstanceofthedialog.
4. Runtheeventloop.
Hereisasamplethatdisplaysamessagebox:
importsys
importgetopt
importgtk
classMessageBox(gtk.Dialog):
def__init__(self,message="",buttons=(),pixmap=None,
modal=True):
gtk.Dialog.__init__(self)
self.connect("destroy",self.quit)
self.connect("delete_event",self.quit)
ifmodal:
self.set_modal(True)
hbox=gtk.HBox(spacing=5)
hbox.set_border_width(5)
self.vbox.pack_start(hbox)
hbox.show()
ifpixmap:
self.realize()
pixmap=Pixmap(self,pixmap)
hbox.pack_start(pixmap,expand=False)
pixmap.show()
label=gtk.Label(message)
hbox.pack_start(label)
label.show()
fortextinbuttons:
b=gtk.Button(text)
b.set_flags(gtk.CAN_DEFAULT)
b.set_data("user_data",text)
b.connect("clicked",self.click)
self.action_area.pack_start(b)
b.show()
Page111
self.ret=None
defquit(self,*args):
self.hide()
self.destroy()
gtk.main_quit()
defclick(self,button):
self.ret=button.get_data("user_data")
self.quit()
#createamessagebox,andreturnwhichbuttonwaspressed
defmessage_box(title="MessageBox",message="",buttons=(),
pixmap=None,
modal=True):
win=MessageBox(message,buttons,pixmap=pixmap,modal=modal)
win.set_title(title)
win.show()
gtk.main()
returnwin.ret
deftest():
result=message_box(title='Test#1',
message='Hereisyourmessage',
buttons=('Ok','Cancel'))
print'result:',result
USAGE_TEXT="""
Usage:
pythonsimple_dialog.py[options]
Options:
Example:
pythonsimple_dialog.py
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
except:
usage()
relink=1
foropt,valinopts:
usage()
iflen(args)!=0:
usage()
test()
main()
Page112
Someexplanation:
First,weimportgtk
NextwedefineaclassMessageBoxthatimplementsamessagebox.Herearea
fewimportantthingstoknowaboutthatclass:
Itisasubclassofgtk.Dialog.
Itcreatesalabelandpacksitintothedialog'sclientarea.NotethataDialogis
aWindowthatcontainsavboxatthetopofandanaction_areaatthebottom
ofitsclientarea.Theintensionisforustopackmiscellaneouswidgetsinto
thevboxandtoputbuttonssuchas"Ok","Cancel",etcintotheaction_area.
Itcreatesonebuttonforeachbuttonlabelpassedtoitsconstructor.The
buttonsareallconnectedtotheclickmethod.
Theclickmethodsavesthevalueoftheuser_dataforthebuttonthatwas
clicked.Inourexample,thisvaluewillbeeither"Ok"or"Cancel".
And,wedefineafunction(message_box)that(1)createsaninstanceofthe
MessageBoxclass,(2)setsitstitle,(3)showsit,(4)startsitseventloopsothatit
cangetandprocesseventsfromtheuser,and(5)returnstheresulttothecaller(in
thiscase"Ok"or"Cancel").
Ourtestingfunction(test)callsfunctionmessage_boxandprintstheresult.
Thislookslikequiteabitofcode,untilyounoticethattheclassMessageBoxand
thefunctionmessage_boxcouldbeputitautilitymoduleandreused.
2.7.2.2Asimpletextinputdialogbox
And,hereisanexamplethatdisplaysantextinputdialog:
importsys
importgetopt
importgtk
classEntryDialog(gtk.Dialog):
def__init__(self,message="",default_text='',modal=True):
gtk.Dialog.__init__(self)
ifmodal:
box=gtk.VBox(spacing=10)
box.set_border_width(10)
self.vbox.pack_start(box)
box.show()
ifmessage:
label=gtk.Label(message)
box.pack_start(label)
label.show()
self.entry=gtk.Entry()
self.entry.set_text(default_text)
box.pack_start(self.entry)
Page113
self.entry.show()
self.entry.grab_focus()
button=gtk.Button("OK")
button.connect("clicked",self.click)
button.set_flags(gtk.CAN_DEFAULT)
self.action_area.pack_start(button)
button.show()
button.grab_default()
button=gtk.Button("Cancel")
button.connect("clicked",self.quit)
button.set_flags(gtk.CAN_DEFAULT)
self.action_area.pack_start(button)
button.show()
self.ret=None
defquit(self,w=None,event=None):
self.hide()
self.destroy()
gtk.main_quit()
defclick(self,button):
self.ret=self.entry.get_text()
self.quit()
definput_box(title="InputBox",message="",default_text='',
modal=True):
win=EntryDialog(message,default_text,modal=modal)
win.show()
gtk.main()
returnwin.ret
deftest():
result=input_box(title='Test#2',
message='Enteravaluexxx:',
default_text='adefaultvalue')
ifresultisNone:
print'Canceled'
else:
print'result:"%s"'%result
USAGE_TEXT="""
Usage:
Options:
Example:
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
Page114
except:
usage()
relink=1
foropt,valinopts:
usage()
iflen(args)!=0:
usage()
test()
main()
Mostoftheexplanationforthemessageboxexampleisrelevanttothisexample,too.
Herearesomedifferences:
OurEntryDialogclassconstructorcreatesinstanceofgtk.Entry,setsitsdefault
value,andpacksitintotheclientarea.
Theconstructoralsoautomaticallycreatestwobuttons:"OK"and"Cancel".The
"OK"buttonisconnecttotheclickmethod,whichsavesthevalueoftheentry
field.The"Cancel"buttonisconnecttothequitmethod,whichdoesnotsavethe
value.
And,ifclassEntryDialogandfunctioninput_boxlookusableanduseful,add
themtoyourutilityguimodule.
2.7.2.3Afileselectiondialogbox
Thisexampleshowsafileselectiondialogbox:
importsys
importgetopt
importgtk
classFileChooser(gtk.FileSelection):
def__init__(self,modal=True,multiple=True):
gtk.FileSelection.__init__(self)
self.multiple=multiple
ifmodal:
self.cancel_button.connect('clicked',self.quit)
self.ok_button.connect('clicked',self.ok_cb)
ifmultiple:
self.set_select_multiple(True)
self.ret=None
defquit(self,*args):
self.hide()
self.destroy()
gtk.main_quit()
Page115
defok_cb(self,b):
ifself.multiple:
self.ret=self.get_selections()
else:
self.ret=self.get_filename()
self.quit()
deffile_sel_box(title="Browse",modal=False,multiple=True):
win=FileChooser(modal=modal,multiple=multiple)
win.show()
gtk.main()
returnwin.ret
deffile_open_box(modal=True):
returnfile_sel_box("Open",modal=modal,multiple=True)
deffile_save_box(modal=True):
returnfile_sel_box("SaveAs",modal=modal,multiple=False)
deftest():
result=file_open_box()
print'openresult:',result
result=file_save_box()
print'saveresult:',result
USAGE_TEXT="""
Usage:
Options:
Example:
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
except:
usage()
relink=1
foropt,valinopts:
usage()
iflen(args)!=0:
usage()
test()
main()
#importpdb
#pdb.run('main()')
Page116
Alittleguidance:
Thereisapredefinedfileselectiondialog.Wesubclassit.
Thisexampledisplaysthefileselectiondialogtwice:oncewithatitle"Open"and
oncewithatitle"SaveAs".
Notehowwecancontrolwhetherthedialogallowsmultiplefileselections.And,
ifweselectthemultipleselectionmode,thenweuseget_selectionsinsteadof
get_filenameinordertogettheselectedfilenames.
Thedialogcontainsbuttonsthatenabletheuserto(1)createanewfolder,(2)
deleteafile,and(3)renameafile.Ifyoudonotwanttheusertoperformthese
operations,thencallhide_fileop_buttons.Thiscalliscommentedoutinour
samplecode.
Notethattherearealsopredefineddialogsforfontselection(FontSelectionDialog)and
colorselection(ColorSelectionDialog)
2.7.3EasyGUI
IfyourGUIneedsareminimalist(maybeapopupdialogortwo)andyourapplicationis
imperativeratherthaneventdriven,thenyoumaywanttoconsiderEasyGUI.Asthe
namesuggests,itisextremelyeasytouse.
HowtoknowwhenyoumightbeabletouseEasyGUI:
Yourapplicationdoesnotneedtoruninawindowcontainingmenusandamenu
bar.
YourGUIneedsamounttolittlemorethandisplayingadialognowandthento
getresponsesfromtheuser.
Youdonotwanttowriteaneventdrivenapplication,thatis,oneinwhichyour
codesitsandwaitsforthetheusertoinitiateoperation,forexample,withmenu
items.
EasyGUIplusdocumentationandexamplesareavailableatEasyGUIhomepageat
SourceForgehttp://easygui.sourceforge.net/
EasyGUIprovidesfunctionsforavarietyofcommonlyneededdialogboxes,including:
Amessageboxdisplaysamessage.
Ayes/nomessageboxdisplays"Yes"and"No"buttons.
Acontinue/cancelmessageboxdisplays"Continue"and"Cancel"buttons.
Achoiceboxdisplaysaselectionlist.
Anenterboxallowsentryofalineoftext.
Anintegerboxallowsentryofaninterger.
Amultipleentryboxallowsentryintomultiplefields.
Codeandtextboxessupportthedisplayoftextinmonospacedorporportional
fonts.
Fileanddirectoryboxesenabletheusertoselectafileoradirectory.
Page117
SeethedocumentationattheEasyGUIWebsiteformorefeatures.
ForademonstrationofEasyGUI'scapabilities,runtheeasygui.pyasaPythonscript:
$pythoneasygui.py
2.7.3.1AsimpleEasyGUIexample
Hereisasimpleexamplethatpromptstheuserforanentry,thenshowstheresponseina
messagebox:
importeasygui
deftesteasygui():
response=easygui.enterbox(msg='Enteryourname:',title='Name
Entry')
easygui.msgbox(msg=response,title='YourResponse')
testeasygui()
2.7.3.2AnEasyGUIfileopendialogexample
Thisexamplepresentsadialogtoallowtheusertoselectafile:
importeasygui
deftest():
response=easygui.fileopenbox(msg='Selectafile')
print'filename:%s'%response
test()
2.8GuidanceonPackagesandModules
2.8.1Introduction
Pythonhasanexcellentrangeofimplementationorganizationstructures.Theserange
fromstatementsandcontrolstructures(atalowlevel)throughfunctions,methods,and
classes(atanintermediatelevel)andmodulesandpackagesatanupperlevel.
Thissectionprovidessomeguidancewiththeuseofpackages.Inparticular:
Howtoconstructandimplementthem.
Howtousethem.
Howtodistributeandinstallthem.
2.8.2ImplementingPackages
APythonpackageisacollectionofPythonmodulesinadiskdirectory.
Page118
Inordertobeabletoimportindividualmodulesfromadirectory,thedirectorymust
containafilenamed__init__.py.(Notethatrequirementdoesnotapplytodirectoriesthat
arelistedinPYTHONPATH.)The__init__.pyservesseveralpurposes:
Thepresenceofthefile__init__.pyinadirectorymarksthedirectoryasaPython
package,whichenablesimportingmodulesfromthedirectory.
Thefirsttimeanapplicationimportsanymodulefromthedirectory/package,the
codeinthemodule__init__isevaluated.
Ifthepackageitselfisimported(asopposedtoanindividualmodulewithinthe
directory/package),thenitisthe__init__thatisimported(andevaluated).
2.8.3UsingPackages
Onesimplewaytoenabletheusertoimportanduseapackageistoinstructtheuseto
importindividualmodulesfromthepackage.
Asecond,slightlymoreadvancedwaytoenabletheusertoimportthepackageisto
exposethosefeaturesofthepackageinthe__init__module.Supposethatmodulemod1
containsfunctionsfun1aandfun1bandsupposethatmodulemod2containsfunctions
fun2aandfun2b.Thenfile__init__.pymightcontainthefollowing:
frommod1importfun1a,fun1b
frommod2importfun2a,fun2b
Then,ifthefollowingisevaluatedintheuser'scode:
importtestpackages
Thentestpackageswillcontainfun1a,fun1b,fun2a,andfun2b.
Forexample,hereisaninteractivesessionthatdemostratesimportingthepackage:
>>>importtestpackages
>>>printdir(testpackages)
[`__builtins__',`__doc__',`__file__',`__name__',
`__path__',
`fun1a',`fun1b',`fun2a',`fun2b',`mod1',`mod2']
2.8.4DistributingandInstallingPackages
Distutils(PythonDistributionUtilities)hasspecialsupportfordistrubutingandinstalling
packages.Learnmorehere:DistributingPythonModules
http://docs.python.org/distutils/index.html.
Asourexample,imaginethatwehaveadirectorycontainingthefollowing:
Testpackages
Testpackages/README
Testpackages/MANIFEST.in
Testpackages/setup.py
Testpackages/testpackages/__init__.py
Page119
Testpackages/testpackages/mod1.py
Testpackages/testpackages/mod2.py
NoticethesubdirectoryTestpackages/testpackagescontainingthefile__init__.py.
ThisisthePythonpackagethatwewillinstall.
We'lldescribehowtoconfiguretheabovefilessothattheycanbepackagedasasingle
distributionfileandsothatthePythonpackagetheycontaincanbeinstalledasapackage
byDistutils.
TheMANIFEST.infileliststhefilesthatwewantincludedinourdistribution.Hereis
thecontentsofourMANIFEST.infile:
includeREADMEMANIFESTMANIFEST.in
includesetup.py
includetestpackages/*.py
Thesetup.pyfiledescribestoDistutils(1)howtopackagethedistributionfileand(2)
howtoinstallthedistribution.Hereisthecontentsofoursamplesetup.py:
fromdistutils.coreimportsetup#[1]
long_description='TestsforinstallinganddistributingPython
packages'
setup(name='testpackages',#[2]
version='1.0a',
description='TestsforPythonpackages',
maintainer='DaveKuhlman',
maintainer_email='dkuhlman@rexx.com',
url='http://www.rexx.com/~dkuhlman',
long_description=long_description,
packages=['testpackages']#[3]
)
Explanation:
1. WeimportthenecessarycomponentfromDistutils.
2. Wedescribethepackageanditsdeveloper/maintainer.
3. Wespecifythedirectorythatistobeinstalledasapackage.Whentheuser
installsourdistribution,thisdirectoryandallthemodulesinitwillbeinstalledas
apackage.
Now,tocreateadistributionfile,werunthefollowing:
pythonsetup.pysdistformats=gztar
whichwillcreateafiletestpackages1.0a.tar.gzunderthedirectorydist.
Then,youcangivethisdistributionfiletoapotentialuser,whocaninstallitbydoingthe
following:
Page120
$tarxvzftestpackages1.0a.tar.gz
$cdtestpackages1.0a
$pythonsetup.pybuild
$pythonsetup.pyinstall#asroot
2.9EndMatter
2.9.1AcknowledgementsandThanks
ThankstotheimplementorsofPythonforproducinganexceptionallyusableand
enjoyableprogramminglanguage.
ThankstoDaveBeazleyandothersforSWIGandPLY.
ThankstoGregEwingforPyrexandPlex.
ThankstoJamesHenstridgeforPyGTK.
2.9.2SeeAlso
ThemainPythonWebSitehttp://www.python.orgformoreinformationon
Python.
PythonDocumentationhttp://www.python.org/doc/forlotsofdocumentation
onPython
Dave'sWebSitehttp://www.rexx.com/~dkuhlmanformoresoftwareand
informationonusingPythonforXMLandtheWeb.
TheSWIGhomepagehttp://www.swig.orgformoreinformationonSWIG
(SimplifiedWrapperandInterfaceGenerator).
ThePyrexhomepagehttp://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
formoreinformationonPyrex.
PLY(PythonLexYacc)homepagehttp://www.dabeaz.com/ply/formore
informationonPLY.
ThePlexhomepagehttp://www.cosc.canterbury.ac.nz/greg.ewing/python/Plex/
formoreinformationonPlex.
DistributingPythonModuleshttp://docs.python.org/distutils/index.htmlfor
informationonthePythonDistributionUtilities(Distutils).
Page121
3Part3PythonWorkbook
3.1Introduction
ThisdocumenttakesaworkbookandexercisewithsolutionsapproachtoPython
training.Itishopedthatthosewhofeelaneedforlessexplanationandmorepractical
exerciseswillfindthisuseful.
Afewnotesabouttheexercises:
I'vetriedtoincludesolutionsformostoftheexercises.Hopefully,youwillbe
abletocopyandpastethesesolutionsintoyourtexteditor,thenextendand
experimentwiththem.
IusetwointeractivePythoninterpreters(althoughtheyarethesamePython
underneath).Whenyouseethisprompt>>>,it'sthestandardPythoninterpreter.
And,whenyouseethispromptIn[1]:,it'sIPython
http://ipython.scipy.org/moin/.
ThelatestversionofthisdocumentisatmyWebsite(URLabove).
Ifyouhavecommentsorsuggestions,pleasesendthemmyway.
3.2LexicalStructures
3.2.1Variablesandnames
Anameisanycombinationofletters,digits,andtheunderscore,butthefirstcharacter
mustbealetteroranunderscore.Namesmaybeofanylength.
Caseissignificant.
Exercises:
1. Whichofthefollowingarevalidnames?
1. total
2. total_of_all_vegetables
3. bigtitle1
4. _inner_func
5. 1bigtitle
6. bigtitle1
2. Whichorthefollowingpairsarethesamename:
1. the_last_itemandthe_last_item
2. the_last_itemandThe_Last_Item
3. itemianditemj
Page122
4. item1anditeml
Solutions:
1. Items1,2,4,and6arevalid.Item3isnotasinglename,butisthreeitems
separatedbytheminusoperator.Item5isnotvalidbecauseitbeginswithadigit.
2. Pythonnamesarecasesensitive,whichmeans:
1. the_last_itemandthe_last_itemarethesame.
2. the_last_itemandThe_Last_ItemaredifferentThesecondname
hasanuppercasecharacters.
3. itemianditemjaredifferent.
4. item1anditemlaredifferentThisonemaybedifficulttosee,
dependingonthefontyouareviewing.Onenameendswiththedigitone;the
otherendswiththealphacharacter"el".Andthisexampleprovidesagood
reasontouse"1"and"l"judiciouslyinnames.
ThefollowingarekeywordsinPythonandshouldnotbeusedasvariablenames:
anddelfromnotwhile
aselifglobalorwith
assertelseifpassyield
breakexceptimportprint
classexecinraise
continuefinallyisreturn
defforlambdatry
Exercises:
1. WhichofthefollowingarevalidnamesinPython?
1. _global
2. global
3. file
Solutions:
1. Donotusekeywordsforvariablenames:
1. Valid
2. Notavalidname."global"isakeyword.
3. Valid,however,"file"isthenameofabuiltintype,asyouwilllearnlater,so
youareadvisednottoredefineit.Hereareafewofthenamesofbuiltin
types:"file","int","str","float","list","dict",etc.SeeBuiltinTypes
http://docs.python.org/lib/types.htmlformorebuiltintypes..
ThefollowingareoperatorsinPythonandwillseparatenames:
+***///%
<<>>&|^~
<><=>===!=<>
andorisnotin
Also:()[].(dot)
Page123
But,notethatthePythonstyleguidesuggeststhatyouplaceblanksaroundbinary
operators.Oneexceptiontothisruleisfunctionargumentsandparametersforfunctions:
itissuggestedthatyounotputblanksaroundtheequalsign(=)usedtospecifykeyword
argumentsanddefaultparameters.
Exercises:
1. Whichofthefollowingaresinglenamesandwhicharenamesseparatedby
operators?
1. fruit_collection
2. fruitcollection
Solutions:
1. Donotuseadash,orotheroperator,inthemiddleofaname:
1. fruit_collectionisasinglename
2. fruitcollectionistwonamesseparatedbyadash.
3.2.2Linestructure
InPython,normallywewriteonestatementperline.Infact,Pythonassumesthis.
Therefore:
Statementseparatorsarenotnormallyneeded.
But,ifwewantmorethanonestatementonaline,weuseastatementseparator,
specificallyasemicolon.
And,ifwewanttoextendastatementtoasecondorthirdlineandsoon,we
sometimesneedtodoabitextra.
ExtendingaPythonstatementtoasubsequentlineFollowthesetworules:
1. Ifthereisanopencontext,nothingspecialneedbedonetoextendastatement
acrossmultiplelines.Anopencontextisanopenparenthesis,anopensquare
bracket,oranopencurlybracket.
2. Wecanalwaysextendastatementonafollowinglinebyplacingabackslashas
thelastcharacteroftheline.
Exercises:
1. Extendthefollowingstatementtoasecondlineusingparentheses:
total_count=tree_count+vegetable_count+
fruit_count
2. Extendthefollowingstatementtoasecondlineusingthebackslashline
continuationcharacter:
total_count=tree_count+vegetable_count+
fruit_count
Solutions:
1. ParenthesescreateanopencontextthattellsPythonthatastatementextendsto
thenextline:
total_count=(tree_count+
Page124
vegetable_count+fruit_count)
2. AbackslashasthelastcharacteronlinetellsPythonthatthecurrentstatement
extendstothenextline:
total_count=tree_count+\
vegetable_count+fruit_count
Forextendingalineonasubsequentline,whichisbetter,parenthesesorabackslash?
Hereisaquote:
"ThepreferredwayofwrappinglonglinesisbyusingPython'simplied
linecontinuationinsideparentheses,bracketsandbraces.Ifnecessary,
youcanaddanextrapairofparenthesesaroundanexpression,but
sometimesusingabackslashlooksbetter."
PEP8:StyleGuideforPythonCode
http://www.python.org/dev/peps/pep0008/
3.2.3Indentationandprogramstructure
Pythonusesindentationtoindicateprogramstructure.Thatistosay,inordertonesta
blockofcodeinsideacompoundstatement,youindentthatnestedcode.Thisisdifferent
frommanyprogramminglanguageswhichusesomesortofbeginandendmarkers,for
examplecurlybrackets.
ThestandardcodingpracticeforPythonistousefourspacesperindentationlevelandto
notusehardtabs.(SeetheStyleGuideforPythonCode.)Becauseofthis,youwillwant
touseatexteditorthatyoucanconfiguresothatitwillusefourspacesforindentation.
SeehereforalistofPythonfriendlytexteditors:PythonEditors.
Exercises:
1. Giventhefollowing,nesttheprintstatementinsidetheifstatement:
ifx>0:
printx
2. Nestthesetwolines:
z=x+y
printz
insidethefollowingfunctiondefinitionstatement:
defshow_sum(x,y):
Solutions:
1. Indentationindicatesthatonestatementisnestedinsideanotherstatement:
ifx>0:
printx
2. Indentationindicatesthatablockofstatementsisnestedinsideanotherstatement:
defshow_sum(x,y):
z=x+y
printz
Page125
3.3ExecutionModel
Hereareafewrules:
1. PythonevaluatesPythoncodefromthetopofamoduledowntothebottomofa
module.
2. Bindingstatementsattoplevelcreatenames(andbindvaluestothosenames)as
Pythonevaluatescode.Furthermore,anameisnotcreateduntilitisboundtoa
value/object.
3. Anestedreferencetoaname(forexample,insideafunctiondefinitionorinthe
nestedblockofanifstatement)isnotuseduntilthatnestedcodeisevaluated.
Exercises:
1. Willthefollowingcodeproduceanerror?
show_version()
defshow_version():
print'Version1.0a'
2. Willthefollowingcodeproduceanerror?
deftest():
show_version()
defshow_version():
print'Version1.0a'
test()
3. Willthefollowingcodeproduceanerror?Assumethatshow_configisnot
defined:
x=3
ifx>5:
show_config()
Solutions:
1. Answer:Yes,itgeneratesanerror.Thenameshow_versionwouldnotbe
createdandboundtoavalueuntilthedeffunctiondefinitionstatementbindsa
functionobjecttoit.Thatisdoneaftertheattempttouse(call)thatobject.
2. Answer:No.Thefunctiontest()doescallthefunctionshow_version(),
butsincetest()isnotcalleduntilaftershow_version()isdefined,thatis
OK.
3. Answer:No.It'sbadcode,butinthiscasewillnotgenerateanerror.Sincexis
lessthan5,thebodyoftheifstatementisnotevaluated.
N.B.Thisexampleshowswhyitisimportantduringtestingthateverylineof
codeinyourPythonprogrambeevaluated.HereisgoodPythonicadvice:"Ifit's
nottested,it'sbroken."
3.4BuiltinDataTypes
Eachofthesubsectionsinthissectiononbuiltindatatypeswillhaveasimilarstructure:
Page126
1. Abriefdescriptionofthedatatypeanditsuses.
2. RepresentationandconstructionHowtorepresentaninstanceofthedatatype.
Howtocodealiteralrepresentationthatcreatesanddefinesaninstance.Howto
createaninstanceofthebuiltintype.
3. Operatorsthatareapplicabletothedatatype.
4. Methodsimplementedandsupportedbythedatatype.
3.4.1Numbers
Thenumbersyouwillusemostcommonlyarelikelytobeintegersandfloats.Python
alsohaslongintegersandcomplexnumbers.
Afewfactsaboutnumbers(inPython):
Pythonwillconverttousingalongintegerautomaticallywhenneeded.Youdo
notneedtoworryaboutexceedingthesizeofa(standard)integer.
ThesizeofthelargestintegerinyourversionofPythonisinsys.maxint.To
learnwhatitis,do:
>>>importsys
>>>printsys.maxint
9223372036854775807
Theaboveshowthemaximumsizeofanintegerona64bitversionofPython.
Youcanconvertfromintegertofloatbyusingthefloatconstructor.Example:
>>>x=25
>>>y=float(x)
>>>printy
25.0
Pythondoes"mixedarithmetic".Youcanadd,multiply,anddivideintegersand
floats.Whenyoudo,Python"promotes"theresulttoafloat.
3.4.1.1Literalrepresentationsofnumbers
Anintegerisconstructedwithaseriesofdigitsortheintegerconstructor(int(x)).Be
awarethatasequenceofdigitsbeginningwithzerorepresentsanoctalvalue.Examples:
>>>x1=1234
>>>x2=int('1234')
>>>x3=25
>>>x1
1234
>>>x2
1234
>>>x3
25
Afloatisconstructedeitherwithdigitsandadot(example,12.345)orwith
engineering/scientificnotationorwiththefloatconstructor(float(x)).Examples:
>>>x1=2.0e3
>>>x1=1.234
Page127
>>>x2=1.234
>>>x3=float('1.234')
>>>x4=2.0e3
>>>x5=2.0e3
>>>printx1,x2,x3,x4,x5
1.2341.2341.2342000.00.002
Exercises:
Constructthesenumericvalues:
1.
2.
3.
4.
5.
6.
7.
Integerzero
Floatingpointzero
Integeronehundredandone
Floatingpointonethousand
Floatingpointonethousandusingscientificnotation
Createapositiveinteger,anegativeinteger,andzero.Assignthemtovariables
Writeseveralarithmeticexpressions.Bindthevaluestovariables.Useavariety
ofoperators,e.g.+,,/,*,etc.Useparenthesestocontroloperatorscope.
8. Createseveralfloatsandassignthemtovariables.
9. Writeseveralarithmeticexpressionscontainingyourfloatvariables.
10. Writeseveralexpressionsusingmixedarithmetic(integersandfloats).Obtaina
floatasaresultofdivisionofoneintegerbyanother;dosobyexplicitly
convertingoneintegertoafloat.
Solutions:
1.
2.
3.
4.
5.
6.
0
0.0,0.,or.0
101
1000.0
1e3or1.0e3
Asigningintegervaluestovariables:
In[7]:value1=23
In[8]:value2=14
In[9]:value3=0
In[10]:value1
Out[10]:23
In[11]:value2
Out[11]:14
In[12]:value3
Out[12]:0
7. Assigningexpressionvaluestovariables:
value1=4*(3+5)
value2=(value1/3.0)2
8. Assigningfloatstovariables:
value1=0.01
value2=3.0
value3=3e4
9. Assigningexpressionscontainingvarialbes:
Page128
value4=value1*(value2value3)
value4=value1+value2+value3value4
10. Mixedarithmetic:
x=5
y=8
z=float(x)/y
Youcanalsoconstructintegersandfloatsusingtheclass.Callingaclass(using
parenthesesafteraclassname,forexample)producesaninstanceoftheclass.
Exercises:
1. Constructanintegerfromthestring"123".
2. Constructafloatfromtheinteger123.
3. Constructanintegerfromthefloat12.345.
Solutions:
1. Usetheintdatatypetoconstructanintegerinstancefromastring:
int("123")
2. Usethefloatdatatypetoconstructafloatinstancefromaninteger:
float(123)
3. Usetheintdatatypetoconstructanintegerinstancefromafloat:
int(12.345)#>12
Noticethattheresultistruncatedtotheintegerpart.
3.4.1.2Operatorsfornumbers
Youcanusemostofthefamiliaroperatorswithnumbers,forexample:
+***///%
<<>>&|^~
<><=>===!=<>
Lookhereforanexplanationoftheseoperatorswhenappliedtonumbers:Numeric
Typesint,float,long,complexhttp://docs.python.org/lib/typesnumeric.html.
Someoperatorstakeprecedenceoverothers.ThetableintheWebpagejustreferenced
abovealsoshowsthatorderofpriority.
Hereisabitofthattable:
Allnumerictypes(exceptcomplex)supportthefollowingoperations,
sortedbyascendingpriority(operationsinthesameboxhavethe
same
priority;allnumericoperationshaveahigherprioritythan
comparison
operations):
OperationResult
x+ysumofxandy
xydifferenceofxandy
Page129
x*yproductofxandy
x/yquotientofxandy
x//y(floored)quotientofxandy
x%yremainderofx/y
xxnegated
+xxunchanged
abs(x)absolutevalueormagnitudeofx
int(x)xconvertedtointeger
long(x)xconvertedtolonginteger
float(x)xconvertedtofloatingpoint
complex(re,im)acomplexnumberwithrealpartre,imaginarypart
im.imdefaultstozero.
c.conjugate()conjugateofthecomplexnumberc
divmod(x,y)thepair(x//y,x%y)
pow(x,y)xtothepowery
x**yxtothepowery
Noticealsothatthesameoperatormayperformadifferentfunctiondependingonthe
datatypeofthevaluetowhichitisapplied.
Exercises:
1. Addthenumbers3,4,and5.
2. Add2totheresultofmultiplying3by4.
3. Add2plus3andmultiplytheresultby4.
Solutions:
1. Arithmeticexpressionsarefollowstandardinfixalgebraicsyntax:
3+4+5
2. Useanotherinfixexpression:
2+3*4
Or:
2+(3*4)
But,inthiscasetheparenthesesarenotnecessarybecausethe*operatorbinds
moretightlythanthe+operator.
3. Useparenthesestocontrolorderofevaluation:
(2+3)*4
Notethatthe*operatorhasprecedenceover(bindstighterthan)the+operator,
sotheparenthesesareneeded.
Pythondoesmixedarithemetic.Whenyouapplyanoperationtoanintegerandafloat,it
promotestheresulttothe"higher"datatype,afloat.
Ifyouneedtoperformanoperationonseveralintegers,butwantuseafloatingpoint
operation,firstconvertoneoftheintegerstoafloatusingfloat(x),whicheffectively
createsaninstanceofclassfloat.
TrythefollowingatyourPythoninteractiveprompt:
1. 1.0+2
2. 2/3Noticethattheresultistruncated.
Page130
3. float(2)/3Noticethattheresultisnottruncated.
Exercises:
1. Giventhefollowingassignments:
x=20
y=50
Dividexbyygivingafloatresult.
Solutions:
1. Promoteoneoftheintegerstofloatbeforeperformingthedivision:
z=float(x)/y
3.4.1.3Methodsonnumbers
Mostofthemethodsimplementedbythedatatypes(classes)intandfloatarespecial
methodsthatarecalledthroughtheuseofoperators.Specialmethodsoftenhavenames
thatbeginandendwithadoubleunderscore.Toseealistofthespecialnamesandabit
ofanindicationofwheneachiscalled,doanyofthefollowingatthePythoninteractive
prompt:
>>>help(int)
>>>help(32)
>>>help(float)
>>>help(1.23)
>>>dir(1)
>>>dir(1.2)
3.4.2Lists
Listsareacontainerdatatypethatactsasadynamicarray.Thatistosay,alistisa
sequencethatcanbeindexedintoandthatcangrowandshrink.
Atupleisanindexablecontainer,likealist,exceptthatatupleisimmutable.
Afewcharacteristicsoflistsandtuples:
Alisthasa(current)lengthGetthelengthofalistwithlen(mylist).
AlisthasanorderTheitemsinalistareordered,andyoucanthinkofthat
orderasgoingfromlefttoright.
AlistisheterogeousYoucaninsertdifferenttypesofobjectsintothesamelist.
Listsaremutable,buttuplesarenot.Thus,thefollowingaretrueoflists,butnot
oftuples:
Youcanextendedoraddtoalist.
Youcanshrinkalistbydeletingitemsfromit.
Youcaninsertitemsintothemiddleofalistoratthebeginningofalist.You
canadditemstotheendofalist.
Youcanchangewhichitemisatagivenpositioninalist.
Page131
3.4.2.1Literalrepresentationoflists
Theliteralrepresentationofalistissquarebracketscontainingzeroormoreitems
separatedbycommas.
Examples:
1. TrytheseatthePythoninteractiveprompt:
>>>[11,22,33]
>>>['aa','bb','cc',]
>>>[100,'apple',200,'banana',]#Thelastcomma
is
>>>optional.
2. Alistcancontainlists.Infactalistcancontainanykindofobject:
>>>[1,[2,3],4,[5,6,7,],8]
3. Listsareheterogenous,thatis,differentkindsofobjectscanbeinthesamelist.
Hereisalistthatcontainsanumber,astring,andanotherlist:
>>>[123,'abc',[456,789]]
Exercises:
1. Create(define)thefollowingtuplesandlistsusingaliteral:
1. Atupleofintegers
2. Atupleofstrings
3. Alistofintegers
4. Alistofstrings
5. Alistoftuplesortupleoflists
6. Alistofintegersandstringsandtuples
7. Atuplecontainingexactlyoneitem
8. Anemptytuple
2. Doeachofthefollowing:
1. Printthelengthofalist.
2. PrinteachiteminthelistIterateovertheitemsinoneofyourlists.Print
eachitem.
3. Appendanitemtoalist.
4. Insertanitematthebeginningofalist.Insertaniteminthemiddleofalist.
5. Addtwoliststogether.Dosobyusingboththeextendmethodandtheplus
(+)operator.Whatisthedifferencebetweenextendingalistandaddingtwo
lists?
6. Retrievethe2nditemfromoneofyourtuplesorlists.
7. Retrievethe2nd,3rd,and4thitems(aslice)fromoneofyourtuplesorlists.
8. Retrievethelast(rightmost)iteminoneofyourlists.
9. Replaceaniteminalistwithanewitem.
10. Poponeitemofftheendofyourlist.
11. Deleteanitemfromalist.
12. Dothefollowinglistmanipulations:
1. Writeafunctionthattakestwoarguments,alistandanitem,andthat
Page132
Solutions:
appendstheitemtothelist.
2. Createanemptylist,
3. Callyourfunctionseveraltimestoappenditemstothelist.
4. Then,printouteachiteminthelist.
1. WecandefinelistliteralsatthePythonorIPythoninteractiveprompt:
1. Createatupleusingcommas,optionallywithparentheses:
In[1]:a1=(11,22,33,)
In[2]:a1
Out[2]:(11,22,33)
2. Quotedcharactersseparatedbycommascreateatupleofstrings:
In[3]:a2=('aaa','bbb','ccc')
In[4]:a2
Out[4]:('aaa','bbb','ccc')
3. Itemsseparatedbycommasinsidesquarebracketscreatealist:
In[26]:a3=[100,200,300,]
In[27]:a3
Out[27]:[100,200,300]
4. Stringsseparatedbycommasinsidesquarebracketscreatealistofstrings:
In[5]:a3=['basil','parsley','coriander']
In[6]:a3
Out[6]:['basil','parsley','coriander']
In[7]:
5. Atupleoralistcancontaintuplesandlists:
In[8]:a5=[(11,22),(33,44),(55,)]
In[9]:a5
Out[9]:[(11,22),(33,44),(55,)]
6. Alistortuplecancontainitemsofdifferenttypes:
In[10]:a6=[101,102,'abc',"def",(201,202),
('ghi','jkl')]
In[11]:a6
Out[11]:[101,102,'abc','def',(201,202),
('ghi','jkl')]
7. Inordertocreateatuplecontainingexactlyoneitem,wemustuseacomma:
In[13]:a7=(6,)
In[14]:a7
Out[14]:(6,)
8. Inordertocreateanemptytuple,usethetupleclass/typetocreateaninstance
ofaemptytuple:
In[21]:a=tuple()
In[22]:a
Out[22]:()
In[23]:type(a)
Out[23]:<type'tuple'>
3.4.2.2Operatorsonlists
Thereareseveraloperatorsthatareapplicabletolists.Hereishowtofindoutabout
Page133
them:
Dodir([])ordir(any_list_instance).Someoftheitemswith
specialnames(leadingandtrainingdoubleunderscores)willgiveyoucluesabout
operatorsimplementedbythelisttype.
Dohelp([])orhelp(list)atthePythoninteractiveprompt.
Dohelp(any_list_instance.some_method),wheresome_method
isoneoftheitemslistedusingdir(any_list_instance).
SeeSequenceTypesstr,unicode,list,tuple,buffer,xrange
http://docs.python.org/lib/typesseq.html
Exercises:
1. Concatenate(add)twoliststogether.
2. Createasinglelistthatcontainstheitemsinaninitiallistrepeated3times.
3. Comparetwolists.
Solutions:
1. Theplusoperator,appliedtotwolistsproducesanewlistthatisaconcatenation
oftwolists:
>>>[11,22]+['aa','bb']
2. Multiplyingalistbyanintegerncreatesanewlistthatrepeatstheoriginallistn
times:
>>>[11,'abc',4.5]*3
3. Thecomparisonoperatorscanbeusedtocomparelists:
>>>[11,22]==[11,22]
>>>[11,22]<[11,33]
3.4.2.3Methodsonlists
Again,usedir()andhelp()tolearnaboutthemethodssupportedbylists.
Examples:
1. Createtwo(small)lists.Extendthefirstlistwiththeitemsinthesecond.
2. Appendseveralindividualitemstotheendofalist.
3. (a)Insertaitematthebeginningofalist.(b)Insertanitemsomewhereinthe
middleofalist.
4. Popanitemofftheendofalist.
Solutions:
1. Theextendmethodaddselementsfromanotherlist,orotheriterable:
>>>a=[11,22,33,44,]
>>>b=[55,66]
>>>a.extend(b)
>>>a
[11,22,33,44,55,66]
2. Usetheappendmethodonalisttoadd/appendanitemtotheendofalist:
>>>a=['aa',11]
Page134
>>>a.append('bb')
>>>a.append(22)
>>>a
['aa',11,'bb',22]
3. Theinsertmethodonalistenablesustoinsertitemsatagivenpositionina
list:
>>>a=[11,22,33,44,]
>>>a.insert(0,'aa')
>>>a
['aa',11,22,33,44]
>>>a.insert(2,'bb')
>>>a
['aa',11,'bb',22,33,44]
But,notethatweuseappendtoadditemsattheendofalist.
4. Thepopmethodonalistreturnsthe"rightmost"itemfromalistandremoves
thatitemfromthelist:
>>>a=[11,22,33,44,]
>>>
>>>b=a.pop()
>>>a
[11,22,33]
>>>b
44
>>>b=a.pop()
>>>a
[11,22]
>>>b
33
Notethattheappendandpopmethodstakentogethercanbeusedtoimplement
astack,thatisaLIFO(lastinfirstout)datastructure.
3.4.2.4Listcomprehensions
Alistcomprehensionisaconvenientwaytoproducealistfromaniterable(asequence
orotherobjectthatcanbeiteratedover).
Initssimplestform,alistcomprehensionresemblestheheaderlineofaforstatement
insidesquarebrackets.However,inalistcomprehension,theforstatementheaderis
prefixedwithanexpressionandsurroundedbysquarebrackets.Hereisatemplate:
[expr(x)forxiniterable]
where:
expr(x)isanexpression,usually,butnotalways,containingx.
iterableissomeiterable.Aniterablemaybeasequence(forexample,alist,a
string,atuple)oranunorderedcollectionoraniterator(somethingoverwhichwe
caniterateorapplyaforstatementto).
Hereisanexample:
Page135
>>>a=[11,22,33,44]
>>>b=[x*2forxina]
>>>b
[22,44,66,88]
Exercises:
1. Giventhefollowinglistofstrings:
names=['alice','bertrand','charlene']
producethefollowinglists:(1)alistofalluppercasenames;(2)alistof
capitalized(firstletteruppercase);
2. Giventhefollowingfunctionwhichcalculatesthefactorialofanumber:
deft(n):
ifn<=1:
returnn
else:
returnn*t(n1)
andthefollowinglistofnumbers:
numbers=[2,3,4,5]
createalistofthefactorialsofeachofthenumbersinthelist.
Solutions:
1. Forourexpressioninalistcomprehension,usetheupperandcapitalize
methods:
>>>names=['alice','bertrand','charlene']
>>>[name.upper()fornameinnames]
['ALICE','BERTRAND','CHARLENE']
>>>[name.capitalize()fornameinnames]
['Alice','Bertrand','Charlene']
2. Theexpressioninourlistcomprehensioncallsthefactorialfunction:
deft(n):
ifn<=1:
returnn
else:
returnn*t(n1)
deftest():
numbers=[2,3,4,5]
factorials=[t(n)forninnumbers]
print'factorials:',factorials
test()
Alistcomprehensioncanalsocontainanifclause.Hereisatemplate:
[expr(x)forxiniterableifpred(x)]
where:
pred(x)isanexpressionthatevaluatestoatrue/falsevalue.Valuesthatcount
asfalsearenumericzero,False,None,andanyemptycollection.Allother
valuescountastrue.
Page136
Onlyvaluesforwhichtheifclauseevaluatestotrueareincludedincreatingtheresulting
list.
Examples:
>>>a=[11,22,33,44]
>>>b=[x*3forxinaifx%2==0]
>>>b
[66,132]
Exercises:
1. Giventwolists,generatealistofallthestringsinthefirstlistthatarenotinthe
secondlist.Herearetwosamplelists:
names1=['alice','bertrand','charlene','daniel']
names2=['bertrand','charlene']
Solutions:
1. Theifclauseofourlistcomprehensionchecksforcontainmentinthelistnames2:
deftest():
names1=['alice','bertrand','charlene',
'daniel']
names2=['bertrand','charlene']
names3=[namefornameinnames1ifnamenotin
names2]
print'names3:',names3
test()
Whenrun,thisscriptprintsoutthefollowing:
names3:['alice','daniel']
3.4.3Strings
Astringisanorderedsequenceofcharacters.Hereareafewcharacteristicsofstrings:
Astringhasalength.Getthelengthwiththelen()builtinfunction.
Astringisindexable.Getasinglecharacteratapositioninastringwiththe
squarebracketoperator,forexamplemystring[5].
Youcanretrieveaslice(substring)ofastringwithasliceoperation,forexample
mystring[5:8].
Createstringswithsinglequotesordoublequotes.Youcanputsinglequotesinside
doublequotesandyoucanputdoublequotesinsidesinglequotes.Youcanalsoescape
characterswithabackslash.
Exercises:
1. Createastringcontainingasinglequote.
2. Createastringcontainingadoublequote.
3. Createastringcontainingbothasinglequoteadoublequote.
Solutions:
Page137
1. Createastringwithdoublequotestoincludesinglequotesinsidethestring:
>>>str1="thatisjerry'sball"
2. Createastringenclosedwithsinglequotesinordertoincludedoublequotes
insidethestring:
>>>str1='say"goodbye",bullwinkle'
3. Takeyourchoice.Escapeeitherthesinglequotesorthedoublequoteswitha
backslash:
>>>str1='say"hello"tojerry\'smom'
>>>str2="say\"hello\"tojerry'smom"
>>>str1
'say"hello"tojerry\'smom'
>>>str2
'say"hello"tojerry\'smom'
Triplequotesenableyoutocreateastringthatspansmultiplelines.Usethreesingle
quotesorthreedoublequotestocreateasinglequotedstring.
Examples:
1. Createatriplequotedstringthatcontainssingleanddoublequotes.
Solutions:
1. Usetriplesinglequotesortripledoublequotestocreatemultilinestrings:
String1='''Thisstringextends
acrossseverallines.And,soithas
endoflinecharactersinit.
'''
String2="""
Thisstringbeginsandendswithanendofline
character.Itcanhaveboth'single'
quotesand"double"quotesinit.
"""
deftest():
printString1
printString2
test()
3.4.3.1Characters
Pythondoesnothaveadistinctcharactertype.InPython,acharacterisastringoflength
1.Youcanusetheord()andchr()builtinfunctionstoconvertfromcharacterto
integerandback.
Exercises:
1. Createacharacter"a".
2. Createacharacter,thenobtainitsintegerrepresentation.
Solutions:
Page138
1. Thecharacter"a"isaplainstringoflength1:
>>>x='a'
2. Theintegerequivalentoftheletter"A":
>>>x="A"
>>>ord(x)
65
3.4.3.2Operatorsonstrings
Youcanconcatenatestringswiththe"+"operator.
Youcancreatemultipleconcatenatedcopiesofastringwiththe"*"operator.
And,augmentedassignment(+=and*=)alsowork.
Examples:
>>>'cat'+'and'+'dog'
'catanddog'
>>>'#'*40
'########################################'
>>>
>>>s1='flower'
>>>s1+='s'
>>>s1
'flowers'
Exercises:
1. Giventhesestrings:
>>>s1='abcd'
>>>s2='efgh'
createanewstringcomposedofthefirststringfollowedby(concatenatedwith)
thesecond.
2. Createasinglestringcontaining5copiesofthestring'abc'.
3. Usethemultiplicationoperatortocreatea"line"of50dashes.
4. Herearethecomponentsofapathtoafileonthefilesystem:"home",
"myusername","Workdir","notes.txt".Concatenatethesetogetherseparating
themwiththepathseparatortoformacompletepathtothatfile.(Notethatifyou
usethebackslashtoseparatecomponentsofthepath,youwillneedtousea
doublebackslash,becausethebackslashistheescapecharacterinstrings.
Solutions:
1. Theplus(+)operatorappliedtoastringcanbeusedtoconcatenatestrings:
>>>s3=s1+s2
>>>s3
'abcdefgh'
2. Themultiplicationoperator(*)appliedtoastringcreatesanewstringthat
concatenatesastringwithitselfsomenumberoftimes:
>>>s1='abc'*5
>>>s1
Page139
'abcabcabcabcabc'
3. Themultiplicationoperator(*)appliedtoastringcanbeusedtocreatea
"horizontaldividerline":
>>>s1=''*50
>>>prints1
4. Thesepmemberoftheosmodulegivesusaplatformindependentwayto
constructpaths:
>>>importos
>>>
>>>a=["home","myusername","Workdir","notes.txt"]
>>>path=a[0]+os.sep+a[1]+os.sep+a[2]+
os.sep+a[3]
>>>path
'home/myusername/Workdir/notes.txt'
And,amoreconcisesolution:
>>>importos
>>>a=["home","myusername","Workdir","notes.txt"]
>>>os.sep.join(a)
'home/myusername/Workdir/notes.txt'
Notes:
Notethatimportingtheosmoduleandthenusingos.sepfromthatmodule
givesusaplatformindependentsolution.
Ifyoudodecidetocodethepathseparatorcharacterexplicitlyandifyouare
onMSWindowswherethepathseparatoristhebackslash,thenyouwillneed
touseadoublebackslash,becausethatcharacteristheescapecharacter.
3.4.3.3Methodsonstrings
Stringsupportavarietyofoperations.Youcanobtainalistofthesemethodsbyusingthe
dir()builtinfunctiononanystring:
>>>dir("")
['__add__','__class__','__contains__','__delattr__','__doc__',
'__eq__','__ge__','__getattribute__','__getitem__',
'__getnewargs__','__getslice__','__gt__','__hash__','__init__',
'__le__','__len__','__lt__','__mod__','__mul__','__ne__',
'__new__','__reduce__','__reduce_ex__','__repr__','__rmod__',
'__rmul__','__setattr__','__str__','capitalize','center',
'count','decode','encode','endswith','expandtabs','find',
'index','isalnum','isalpha','isdigit','islower','isspace',
'istitle','isupper','join','ljust','lower','lstrip',
'partition','replace','rfind','rindex','rjust','rpartition',
'rsplit','rstrip','split','splitlines','startswith','strip',
'swapcase','title','translate','upper','zfill']
And,youcangethelponanyspecificmethodbyusingthehelp()builtinfunction.
Hereisanexample:
>>>help("".strip)
Page140
Helponbuiltinfunctionstrip:
strip(...)
S.strip([chars])>stringorunicode
ReturnacopyofthestringSwithleadingandtrailing
whitespaceremoved.
IfcharsisgivenandnotNone,removecharactersinchars
instead.
Ifcharsisunicode,Swillbeconvertedtounicodebefore
stripping
Exercises:
1. Stripallthewhitespacecharactersofftherightendofastring.
2. Centerashortstringwithinalongerstring,thatis,padashortstringwithblank
charactersonbothrightandlefttocenterit.
3. Convertastringtoalluppercase.
4. Splitastringintoalistof"words".
5. (a)Jointhestringsinalistofstringstoformasinglestring.(b)Ditto,butputa
newlinecharacterbetweeneachoriginalstring.
Solutions:
1. Therstrip()methodstripswhitespaceofftherightsideofastring:
>>>s1='sometext\n'
>>>s1
'sometext\n'
>>>s2=s1.rstrip()
>>>s2
'sometext'
2. Thecenter(n)methodcentersastringwithinapaddedstringofwidthn:
>>>s1='Dave'
>>>s2=s1.center(20)
>>>s2
'Dave'
3. Theupper()methodproducesanewstringthatconvertsallalphacharactersin
theoriginaltouppercase:
>>>s1='Banana'
>>>s1
'Banana'
>>>s2=s1.upper()
>>>s2
'BANANA'
4. Thesplit(sep)methodproducesalistofstringsthatareseparatedbysepin
theoriginalstring.Ifsepisomitted,whitespaceistreatedastheseparator:
>>>s1="""howdoesitfeel
...tobeonyourown
...nodirectionsknown
...likearollingstone
..."""
>>>words=s1.split()
Page141
>>>words
['how','does','it','feel','to','be','on','your',
'own','no',
'directions','known','like','a','rolling','stone']
Notethatthesplit()functioninthere(regularexpression)moduleisuseful
whentheseparatorismorecomplexthanwhitespaceorasinglecharacter.
5. Thejoin()methodconcatenatesstringsfromalistofstringstoformasingle
string:
>>>lines=[]
>>>lines.append('howdoesitfeel')
>>>lines.append('tobeonyourown')
>>>lines.append('nodirectionsknown')
>>>lines.append('likearollingstone')
>>>lines
['howdoesitfeel','tobeonyourown','no
directionsknown',
'likearollingstone']
>>>s1=''.join(lines)
>>>s2=''.join(lines)
>>>s3='\n'.join(lines)
>>>s1
'howdoesitfeeltobeonyourownnodirections
knownlikearollingstone'
>>>s2
'howdoesitfeeltobeonyourownnodirectionsknown
likearollingstone'
>>>s3
'howdoesitfeel\ntobeonyourown\nnodirections
known\nlikearollingstone'
>>>prints3
howdoesitfeel
tobeonyourown
nodirectionsknown
likearollingstone
3.4.3.4Rawstrings
Rawstringsgiveusaconvenientwaytoincludethebackslashcharacterinastring
withoutescaping(withanadditionalbackslash).Rawstringslooklikeplainliteral
strings,butareprefixedwithan"r"or"R".SeeStringliterals
http://docs.python.org/reference/lexical_analysis.html#stringliterals
Excercises:
1. Createastringthatcontainsabackslashcharacterusingbothplainliteralstring
andarawstring.
Solutions:
1. Weusean"r"prefixtodefinearawstring:
>>>print'abc\\def'
abc\def
>>>printr'abc\def'
Page142
abc\def
3.4.3.5Unicodestrings
Unicodestringsgiveusaconsistentwaytoprocesscharacterdatafromavarietyof
characterencodings.
Excercises:
1. Createseveralunicodestrings.Useboththeunicodeprefixcharacter("u")andthe
unicodetype(unicode(some_string)).
2. Convertastring(possiblyfromanothernonasciiencoding)tounicode.
3. Convertaunicodestringtoanotherencoding,forexample,utf8.
4. Testastringtodetermineifitisunicode.
5. Createastringthatcontainsaunicodecharacter,thatis,acharacteroutsidethe
asciicharacterset.
Solutions:
1. Wecanrepresentunicodestringwitheitherthe"u"prefixorwithacalltothe
unicodetype:
defexercise1():
a=u'abcd'
printa
b=unicode('efgh')
printb
2. Weconvertastringfromanothercharacterencodingintounicodewiththe
decode()stringmethod:
importsys
defexercise2():
a='abcd'.decode('utf8')
printa
b='abcd'.decode(sys.getdefaultencoding())
printb
3. Wecanconvertaunicodestringtoanothercharacterencodingwiththe
encode()stringmethod:
importsys
defexercise3():
a=u'abcd'
printa.encode('utf8')
printa.encode(sys.getdefaultencoding())
4. Herearetwowaystocheckthetypeofastring:
importtypes
defexercise4():
a=u'abcd'
printtype(a)istypes.UnicodeType
printtype(a)istype(u'')
5. Wecanencodeunicodecharactersinastringinseveralways,forexample,(1)by
Page143
definingautf8stringandconvertingittounicodeor(2)definingastringwithan
embeddedunicodecharacteror(3)concatenatingaunicodecharacherintoa
string:
defexercise5():
utf8_string='IvanKrsti\xc4\x87'
unicode_string=utf8_string.decode('utf8')
printunicode_string.encode('utf8')
printlen(utf8_string)
printlen(unicode_string)
unicode_string=u'aa\u0107bb'
unicode_string='aa'+unichr(263)+'bb'
Guidanceforuseofencodingsandunicode:
1. Convert/decodefromanexternalencodingtounicodeearly:
my_source_string.decode(encoding)
2. Doyourwork(Pythonprocessing)inunicode.
3. Convert/encodetoanexternalencodinglate(forexample,justbeforesavingtoan
externalfile):
my_unicode_string.encode(encoding)
Formoreinformation,see:
UnicodeInPython,CompletelyDemystifiedhttp://farmdev.com/talks/unicode/
UnicodeHowtohttp://www.amk.ca/python/howto/unicode.
PEP100:PythonUnicodeIntegrationhttp://www.python.org/dev/peps/pep
0100/
4.8codecsCodecregistryandbaseclasseshttp://docs.python.org/lib/module
codecs.html
4.8.2EncodingsandUnicodehttp://docs.python.org/lib/encodings
overview.html
4.8.3StandardEncodingshttp://docs.python.org/lib/standardencodings.html
ConvertingUnicodeStringsto8bitStringshttp://effbot.org/zone/unicode
convert.htm
3.4.4Dictionaries
Adictionaryisanunorderedcollectionofkeyvaluepairs.
Adictionaryhasalength,specificallythenumberofkeyvaluepairs.
Thekeysmustbeimmutableobjecttypes.
3.4.4.1Literalrepresentationofdictionaries
Curleybracketsareusedtorepresentadictionary.Eachpairinthedictionaryis
representedbyakeyandvalueseparatedbyacolon.Multiplepairsareseparatedby
comas.Forexample,hereisanemptydictionaryandseveraldictionariescontaining
Page144
key/valuepairs:
In[4]:d1={}
In[5]:d2={'width':8.5,'height':11}
In[6]:d3={1:'RED',2:'GREEN',3:'BLUE',}
In[7]:d1
Out[7]:{}
In[8]:d2
Out[8]:{'height':11,'width':8.5}
In[9]:d3
Out[9]:{1:'RED',2:'GREEN',3:'BLUE'}
Notes:
Acommaafterthelastpairisoptional.SeetheREDGREENBLUEexample
above.
Stringsandintegersworkaskeys,sincetheyareimmutable.Youmightalsowant
tothinkabouttheuseoftuplesofintegersaskeysinadictionaryusedto
representasparsearray.
Exercises:
1. Defineadictionarythathasthefollowingkeyvaluepairs:
2. Defineadictionarytorepresentthe"enum"daysoftheweek:Sunday,Monday,
Tuesday,...
Solutions:
1. Adictionarywhosekeysandvaluesarestringscanbeusedtorepresentthistable:
vegetables={
'Eggplant':'Purple',
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}
Notethattheopencurlybracketenablesustocontinuethisstatementacross
multiplelineswithoutusingabackslash.
2. Wemightusestringsforthenamesofthedaysoftheweekaskeys:
DAYS={
'Sunday':1,
'Monday':2,
'Tuesday':3,
'Wednesday':4,
'Thrusday':5,
'Friday':6,
'Saturday':7,
}
3.4.4.2Operatorsondictionaries
Dictionariessupportthefollowing"operators":
Lengthlen(d)returnsthenumberofpairsinadictionary.
Page145
IndexingYoucanbothsetandgetthevalueassociatedwithakeybyusingthe
indexingoperator[].Examples:
In[12]:d3[2]
Out[12]:'GREEN'
In[13]:d3[0]='WHITE'
In[14]:d3[0]
Out[14]:'WHITE'
TestforkeyTheinoperatortestsfortheexistenceofakeyinadictionary.
Example:
In[6]:trees={'poplar':'deciduous','cedar':
'evergreen'}
In[7]:if'cedar'intrees:
...:print'Thecedaris%s'%
(trees['cedar'],)
...:
Thecedarisevergreen
Exercises:
1. Createanemptydictionary,thenusetheindexingoperator[]toinsertthe
followingnamevaluepairs:
"red""255:0:0"
"green""0:255:0"
"blue""0:0:255"
2. Printoutthenumberofitemsinyourdictionary.
Solutions:
1. Wecanuse"[]"tosetthevalueofakeyinadictionary:
deftest():
colors={}
colors["red"]="255:0:0"
colors["green"]="0:255:0"
colors["blue"]="0:0:255"
print'Thevalueofredis"%s"'%
(colors['red'],)
print'Thecolorsdictionarycontains%ditems.'%
(len(colors),)
test()
Whenwerunthis,wesee:
Thevalueofredis"255:0:0"
Thecolorsdictionarycontains3items.
2. Thelen()builtinfunctiongivesusthenumberofitemsinadictionary.Seethe
previoussolutionforanexampleofthis.
3.4.4.3Methodsondictionaries
Hereisatablethatdescribesthemethodsapplicabletodictionarys:
Operation
len(a)
Result
thenumberofitemsina
Page146
Operation
Result
a[k]
theitemofawithkeyk
a[k]=v
seta[k]tov
dela[k]
removea[k]froma
a.clear()
removeallitemsfroma
a.copy()
a(shallow)copyofa
kina
Trueifahasakeyk,elseFalse
knotina
equivalenttonotkina
a.has_key(k)
equivalenttokina,usethatforminnewcode
a.items()
acopyofa'slistof(key,value)pair
a.keys()
acopyofa'slistofkeys
a.update([b])
updatesawithkey/valuepairsfromb,overwritingexisting
keys,returnsNone
a.fromkeys(seq[,value])
createsanewdictionarywithkeysfromseqandvaluessetto
value
a.values()
acopyofa'slistofvalues
a.get(k[,x])
a[k]ifkina,elsex)
a.setdefault(k[,x])
a[k]ifkina,elsex(alsosettingit)
a.pop(k[,x])
a[k]ifkina,elsex(andremovek)(8)
a.popitem()
removeandreturnanarbitrary(key,value)pair
a.iteritems()
returnaniteratorover(key,value)pairs
a.iterkeys()
returnaniteratoroverthemapping'skeys
a.itervalues()
returnaniteratoroverthemapping'svalues
YoucanalsofindthistableatthestandarddocumentationWebsiteinthe"Python
LibraryReference":MappingTypesdicthttp://docs.python.org/lib/typesmapping.html
Exercises:
1. Printthekeysandvaluesintheabove"vegetable"dictionary.
2. Printthekeysandvaluesintheabove"vegetable"dictionarywiththekeysin
alphabeticalorder.
Page147
3. Testfortheoccuranceofakeyinadictionary.
Solutions:
1. Wecanusethed.items()methodtoretrievealistoftuplescontainingkey
valuepairs,thenuseunpackingtocapturethekeyandvalue:
Vegetables={
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}
deftest():
forkey,valueinVegetables.items():
print'key:',key,'value:',value
test()
2. Weretrievealistofkeyswiththekeys()method,thesortitwiththelist
sort()method:
Vegetables={
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}
deftest():
keys=Vegetables.keys()
keys.sort()
forkeyinkeys:
print'key:',key,'value:',Vegetables[key]
test()
3. Totestfortheexistenceofakeyinadictionary,wecanuseeitherthein
operator(preferred)orthed.has_key()method(oldstyle):
Vegetables={
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}
deftest():
if'Eggplant'inVegetables:
print'wehave%segplants'%
Vegetables['Eggplant']
if'Banana'notinVegetables:
print'yeswehavenobananas'
ifVegetables.has_key('Parsley'):
print'wehaveleafy,%sparsley'%
Page148
Vegetables['Parsley']
test()
Whichwillprintout:
wehavePurpleegplants
yeswehavenobananas
wehaveleafy,Greenparsley
3.4.5Files
APythonfileobjectrepresentsafileonafilesystem.
Afileobjectopenforreadingatextfileisiterable.Whenweiterateoverit,itproduces
thelinesinthefile.
Afilemaybeopenedinthesemodes:
'r'readmode.Thefilemustexist.
'w'writemode.Thefileiscreated;anexistingfileisoverwritten.
'a'appendmode.Anexistingfileisopenedforwriting(attheendofthefile).A
fileiscreatedifitdoesnotexist.
Theopen()builtinfunctionisusedtocreateafileobject.Forexample,thefollowing
code(1)opensafileforwriting,then(2)forreading,then(3)forappending,andfinally
(4)forreadingagain:
deftest(infilename):
#1.Openthefileinwritemode,whichcreatesthefile.
outfile=open(infilename,'w')
outfile.write('line1\n')
outfile.close()
#2.Openthefileforreading.
forlineininfile:
print'Line:',line.rstrip()
infile.close()
#3.Openthefileinappendmode,andaddalinetotheendof
#thefile.
outfile=open(infilename,'a')
outfile.close()
print''*40
#4.Openthefileinreadmodeoncemore.
forlineininfile:
print'Line:',line.rstrip()
infile.close()
test('tmp.txt')
Exercises:
Page149
1. Openatextfileforreading,thenreadtheentirefileasasinglestring,andthen
splitthecontentonnewlinecharacters.
2. Openatextfileforreading,thenreadtheentirefileasalistofstrings,whereeach
stringisonelineinthefile.
3. Openatextfileforreading,theniterateofeachlineinthefileandprintitout.
Solutions:
1. Usetheopen()builtinfunctiontoopenthefileandcreateafileobject.Usethe
read()methodonthefileobjecttoreadtheentirefile.Usethesplit()or
splitlines()methodstosplitthefileintolines:
>>>infile=open('tmp.txt','r')
>>>content=infile.read()
>>>infile.close()
>>>lines=content.splitlines()
>>>printlines
['line1','line2','line3','']
2. Thef.readlines()methodreturnsalistoflinesinafile:
>>>infile=open('tmp.txt','r')
>>>lines=infile.readlines()
>>>infile.close()
>>>printlines
['line1\n','line2\n','line3\n']
3. Sinceafileobject(openforreading)isitselfaniterator,wecaniterateoveritina
forstatement:
"""
Testiterationoveratextfile.
Usage:
pythontest.pyin_file_name
"""
importsys
forlineininfile:
#Stripoffthenewlinecharacterandany
whitespaceon
#theright.
line=line.rstrip()
#Printonlynonblanklines.
ifline:
printline
infile.close()
defmain():
args=sys.argv[1:]
iflen(args)!=1:
print__doc__
sys.exit(1)
infilename=args[0]
test(infilename)
Page150
main()
Notes:
Thelasttwolinesofthissolutioncheckthe__name__attributeofthe
moduleitselfsothatthemodulewillrunasascriptbutwillnotrunwhenthe
moduleisimportedbyanothermodule.
The__doc__attributeofthemodulegivesusthemodule'sdocstring,which
isthestringdefinedatthetopofthemodule.
sys.argvgivesusthecommandline.And,sys.argv[1:]chopsoffthe
programname,leavinguswiththecommanlinearguments.
3.4.6Afewmiscellaneousdatatypes
3.4.6.1None
Noneisasingleton.ThereisonlyoneinstanceofNone.Usethisvaluetoindicatethe
absenceofanyother"real"value.
TestforNonewiththeidentityoperatoris.
Exercises:
1. Createalist,someofwhoseelementsareNone.Thenwriteaforloopthat
countsthenumberofoccurancesofNoneinthelist.
Solutions:
1. TheidentityoperatorsisandisnotcanbeusedtotestforNone:
>>>a=[11,None,'abc',None,{}]
>>>a
[11,None,'abc',None,{}]
>>>count=0
>>>foritemina:
...ifitemisNone:
...count+=1
...
>>>
>>>printcount
2
3.4.6.2ThebooleansTrueandFalse
PythonhasthetwobooleanvaluesTrueandFalse.Manycomparisonoperatorsreturn
TrueandFalse.
Examples:
1. Whatvalueisreturnedby3>2?
Answer:ThebooleanvalueTrue.
2. Giventhesevariabledefinitions:
Page151
x=3
y=4
z=5
Whatdoesthefollowingprintout:
printy>xandz>y
AnswerPrintsout"True"
3.5Statements
3.5.1Assignmentstatement
Theassignmentstatementusestheassignmentoperator=.
Theassignmentstatementisabindingstatement:itbindsavaluetoanamewithina
namespace.
Exercises:
1. Bindthevalue"eggplant"tothevariablevegetable.
Solutions:
1.The=operatorisanassignmentstatementthatbindsavaluetoavariable:
>>>vegetable="eggplant"
Thereisalsoaugmentedassignmentusingtheoperators+=,=,*=,/=,etc.
Exercises:
1. Useaugmentedassignmenttoincrementthevalueofaninteger.
2. Useaugmentedassignmenttoappendcharacterstotheendofastring.
3. Useaugmentedassignmenttoappendtheitemsinonelisttoanother.
4. Useaugmentedassignmenttodecrementavariablecontaininganintegerby1.
Solutions:
1. The+=operatorincrementsthevalueofaninteger:
>>>count=0
>>>count+=1
>>>count
1
>>>count+=1
>>>count
2
2. The+=operatorappendscharacterstotheendofastring:
>>>buffer='abcde'
>>>buffer+='fgh'
>>>buffer
'abcdefgh'
3. The+=operatorappendsitemsinonelisttoanother:
In[20]:a=[11,22,33]
In[21]:b=[44,55]
Page152
In[22]:a+=b
In[23]:a
Out[23]:[11,22,33,44,55]
1. The=operatordecrementsthevalueofaninteger:
>>>count=5
>>>count
5
>>>count=1
>>>count
4
Youcanalsoassignavalueto(1)anelementofalist,(2)aniteminadictionary,(3)an
attributeofanobject,etc.
Exercises:
1. Createalistofthreeitems,thenassignanewvaluetothe2ndelementinthelist.
2. Createadictionary,thenassignvaluestothekeys"vegetable"and"fruit"inthat
dictionary.
3. Usethefollowingcodetocreateaninstanceofaclass:
classA(object):
pass
a=A()
Thenassignvaluestoanattribuenamedcategoryinthatinstance.
Solutions:
1. Assignmentwiththeindexingoperator[]assignsavaluetoanelementinalist:
>>>trees=['pine','oak','elm']
>>>trees
['pine','oak','elm']
>>>trees[1]='cedar'
>>>trees
['pine','cedar','elm']
2. Assignmentwiththeindexingoperator[]assignsavaluetoanitem(akeyvalue
pair)inadictionary:
>>>foods={}
>>>foods
{}
>>>foods['vegetable']='greenbeans'
>>>foods['fruit']='nectarine'
>>>foods
{'vegetable':'greenbeans','fruit':'nectarine'}
3. Assignmentalongwiththedereferencingoperator.(dot)enablesustoassigna
valuetoanattributeofanobject:
>>>classA(object):
...pass
...
>>>a=A()
>>>a.category=25
>>>a.__dict__
{'category':25}
>>>a.category
Page153
25
3.5.2printstatement
Warning:BeawarethattheprintstatementwillgoawayinPythonversion3.0.Itwill
bereplacedbythebuiltinprint()function.
Theprintstatementsendsoutputtostandardoutput.Itprovidesasomewhatmore
convenientwayofproducingoutputthanusingsys.stdout.write().
Theprintstatementtakesaseriesofzeroormoreobjectsseparatedbycommas.Zero
objectsproducesablankline.
Theprintstatementnormallyaddsanewlineattheendofitsoutput.Toeliminatethat,
addacommaattheend.
Exercises:
1. Printasinglestring.
2. Printthreestringsusingasingleprintstatement.
3. Givenavariablenamecontainingastring,printoutthestringMynameis
"xxxx".,wherexxxxisreplacebythevalueofname.Usethestringformatting
operator.
Solutions:
1. Wecanprintaliteralstring:
>>>print'Hello,there'
Hello,there
2. Wecanprintliteralsandthevalueofvariables:
>>>description='cute'
>>>print'Iama',description,'kid.'
Iamacutekid.
3. Thestringformattingoperatorgivesmorecontroloverformattingoutput:
>>>name='Alice'
>>>print'Mynameis"%s".'%(name,)
Mynameis"Alice".
3.5.3if:statementexercises
Theifstatementisacompoundstatementthatenablesustoconditionallyexecute
blocksofcode.
Theifstatementalsohasoptionalelif:andelse:clauses.
Theconditioninanif:orelif:clausecanbeanyPythonexpression,inotherwords,
somethingthatreturnsavalue(evenifthatvalueisNone).
Intheconditioninanif:orelif:clause,thefollowingvaluescountas"false":
False
None
Page154
Numericzero
Anemptycollection,forexampleanemptylistordictionary
Anemptystring(astringoflengthzero)
Allothervaluescountastrue.
Exercises:
1. Giventhefollowinglist:
>>>bananas=['banana1','banana2','banana3',]
Printonemessageifitisanemptylistandanothermessgeifitisnot.
2. HereisonewayofdefiningaPythonequivalentofan"enum":
NO_COLOR,RED,GREEN,BLUE=range(4)
Writeanif:statementwhichimplementstheeffectofa"switch"statementin
Python.Printoutauniquemessageforeachcolor.
Solutions:
1. Wecantestforanemptyornonemptylist:
>>>bananas=['banana1','banana2','banana3',]
>>>ifnotbananas:
...print'yes,wehavenobananas'
...else:
...print'yes,wehavebananas'
...
yes,wehavebananas
2. Wecansimulatea"switch"statementusingif:elif:...:
NO_COLOR,RED,GREEN,BLUE=range(4)
deftest(color):
ifcolor==RED:
print"It'sred."
elifcolor==GREEN:
print"It'sgreen."
elifcolor==BLUE:
print"It'sblue."
defmain():
color=BLUE
test(color)
main()
Which,whenrunprintsoutthefollowing:
It'sblue.
3.5.4for:statementexercises
Thefor:statementisthePythonwaytoiterateoverandprocesstheelementsofa
collectionorotheriterable.
Thebasicformofthefor:statementisthefollowing:
Page155
forXinY:
statement
o
o
o
where:
Xissomethingthatcanbeassignedto.ItissomethingtowhichPythoncanbinda
value.
Yissomecollectionorotheriterable.
Exercises:
1. Createalistofintegers.Useafor:statementtoprintouteachintegerinthelist.
2. Createastring.printouteachcharacterinthestring.
Solutions:
1. Thefor:statementcaniterateovertheitemsinalist:
In[13]:a=[11,22,33,]
In[14]:forvalueina:
....:print'value:%d'%value
....:
....:
value:11
value:22
value:33
2. Thefor:statementcaniterateoverthecharactersinastring:
In[16]:b='chocolate'
In[17]:forchr1inb:
....:print'character:%s'%chr1
....:
....:
character:c
character:h
character:o
character:c
character:o
character:l
character:a
character:t
character:e
Notes:
Inthesolution,Iusedthevariablenamechr1ratherthanchrsoasnotto
overwritethenameofthebuiltinfunctionchr().
Whenweneedasequentialindex,wecanusetherange()builtinfunctiontocreatea
listofintegers.And,thexrange()builtinfunctionproducesaninteratorthatproduces
asequenceofintegerswithoutcreatingtheentirelist.Toiterateoveralargesequenceof
integers,usexrange()insteadofrange().
Exercises:
1. Printouttheintegersfrom0to5insequence.
Page156
2. Computethesumofalltheintegersfrom0to99999.
3. Giventhefollowinggeneratorfunction:
importurllib
Urls=[
'http://yahoo.com',
'http://python.org',
'http://gimp.org',#TheGNUimagemanipulation
program
]
defwalk(url_list):
forurlinurl_list:
f=urllib.urlopen(url)
stuff=f.read()
f.close()
yieldstuff
Writeafor:statementthatusesthisiteratorgeneratortoprintthelengthsofthe
contentateachoftheWebpagesinthatlist.
Solutions:
1. Therange()builtinfunctiongivesusasequencetoiterateover:
In[5]:foridxinrange(6):
...:print'idx:%d'%idx
...:
...:
idx:0
idx:1
idx:2
idx:3
idx:4
idx:5
2. Sincethatsequenceisabitlarge,we'llusexrange()insteadofrange():
In[8]:count=0
In[9]:forninxrange(100000):
...:count+=n
...:
...:
In[10]:count
Out[10]:4999950000
3. Thefor:statementenablesustoiterateoveriterablesaswellascollections:
importurllib
Urls=[
'http://yahoo.com',
program
]
defwalk(url_list):
forurlinurl_list:
Page157
stuff=f.read()
f.close()
yieldstuff
deftest():
forurlinwalk(Urls):
print'length:%d'%(len(url),)
test()
WhenIranthisscript,itprintsthefollowing:
length:9562
length:16341
length:12343
Ifyouneedanindexwhileiteratingoverasequence,considerusingtheenumerate()
builtinfunction.
Exercises:
1. Giventhefollowingtwolistsofintegersofthesamelength:
a=[1,2,3,4,5]
b=[100,200,300,400,500]
Addthevaluesinthefirstlisttothecorrespondingvaluesinthesecondlist.
Solutions:
1. Theenumerate()builtinfunctiongivesusanindexandvaluesfroma
sequence.Sinceenumerate()givesusaninteratorthatproducesasequenceof
twotuples,wecanunpackthosetuplesintoindexandvaluevariablesinthe
headerlineoftheforstatement:
In[13]:a=[1,2,3,4,5]
In[14]:b=[100,200,300,400,500]
In[15]:
In[16]:foridx,valueinenumerate(a):
....:b[idx]+=value
....:
....:
In[17]:b
Out[17]:[101,202,303,404,505]
3.5.5while:statementexercises
Awhile:statementexecutesablockofcoderepeatedlyaslongasaconditionistrue.
Hereisatemplateforthewhile:statement:
whilecondition:
statement
o
o
o
Where:
Page158
conditionisanexpression.Theexpressionissomethingthatreturnsavalue
whichcanbeinterpretedastrueorfalse.
Exercises:
1. Writeawhile:loopthatdoublesallthevaluesinalistofintegers.
Solutions:
1. Awhile:loopwithanindexvariablecanbeusedtomodifyeachelementofa
list:
deftest_while():
numbers=[11,22,33,44,]
print'before:%s'%(numbers,)
idx=0
whileidx<len(numbers):
numbers[idx]*=2
idx+=1
print'after:%s'%(numbers,)
But,noticethatthistaskiseasierusingthefor:statementandthebuiltin
enumerate()function:
deftest_for():
numbers=[11,22,33,44,]
foridx,iteminenumerate(numbers):
numbers[idx]*=2
3.5.6breakandcontinuestatements
Thecontinuestatementskipstheremainderofthestatementsinthebodyofaloop
andstartsimmediatelyatthetopoftheloopagain.
Abreakstatementinthebodyofaloopterminatestheloop.Itexitsfromthe
immediatelycontainingloop.
breakandcontinuecanbeusedinbothfor:andwhile:statements.
Exercises:
1. Writeafor:loopthattakesalistofintegersandtripleseachintegerthatiseven.
Usethecontinuestatement.
2. Writealoopthattakesalistofintegersandcomputesthesumofalltheintegers
upuntilazeroisfoundinthelist.Usethebreakstatement.
Solutions:
1. Thecontinuestatementenablesusto"skip"itemsthatsatisfyaconditionor
test:
deftest():
numbers=[11,22,33,44,55,66,]
foridx,iteminenumerate(numbers):
ifitem%2!=0:
Page159
continue
numbers[idx]*=3
test()
2. Thebreakstatementenablesustoexitfromaloopwhenwefindazero:
deftest():
numbers=[11,22,33,0,44,55,66,]
print'numbers:%s'%(numbers,)
sum=0
foriteminnumbers:
ifitem==0:
break
sum+=item
print'sum:%d'%(sum,)
test()
3.5.7Exceptionsandthetry:except:andraisestatements
Thetry:except:statementenablesustocatchanexceptionthatisthrownfrom
withinablockofcode,orfromcodecalledfromanydepthwithingthatblock.
Theraisestatementenablesustothrowanexception.
Anexceptionisaclassoraninstanceofanexceptionclass.Ifanexceptionisnotcaught,
itresultsinatracebackandterminationoftheprogram.
Thereisasetofstandardexceptions.Youcanlearnaboutthemhere:BuiltinExceptions
http://docs.python.org/lib/moduleexceptions.html.
Youcandefineyourownexceptionclasses.Todoso,createanemptysubclassofthe
classException.Definingyourownexceptionwillenableyou(orothers)tothrow
andthencatchthatspecificexceptiontypewhileignoreothersexceptions.
Exercises:
1. Writeatry:except:statementthatattemptstoopenafileforreadingand
catchestheexceptionthrownwhenthefiledoesnotexist.
Question:Howdoyoufindoutthenameoftheexceptionthatisthrownforan
input/outputerrorsuchasthefailuretoopenafile?
2. Defineanexceptionclass.Thenwriteatry:except:statementinwhichyou
throwandcatchthatspecificexception.
3. Defineanexceptionclassanduseittoimplementamultilevelbreakfroman
innerloop,bypassinganouterloop.
Solutions:
1. UsethePythoninteractiveinterpretertolearntheexceptiontypethrownwhena
I/Oerroroccurs.Example:
>>>infile=open('xx_nothing__yy.txt','r')
Page160
File"<stdin>",line1,in<module>
IOError:[Errno2]Nosuchfileordirectory:
'xx_nothing__yy.txt'
>>>
Inthiscase,theexceptiontypeisIOError.
Now,writeatry:except:blockwhichcatchesthatexception:
deftest():
infilename='nothing_noplace.txt'
try:
forlineininfile:
printline
exceptIOError,exp:
print'cannotopenfile"%s"'%infilename
test()
2. WedefineaexceptionclassasasubclassofclassException,thenthrowit
(withtheraisestatement)andcatchit(withatry:except:statement):
classSizeError(Exception):
pass
deftest_exception(size):
try:
ifsize<=0:
raiseSizeError,'sizemustbegreaterthan
zero'
#Produceadifferenterrortoshowthatit
willnotbecaught.
x=y
exceptSizeError,exp:
print'%s'%(exp,)
print'goodbye'
deftest():
test_exception(1)
print''*40
test_exception(1)
test()
Whenwerunthisscript,itproducesthefollowingoutput:
sizemustbegreaterthanzero
goodbye
File"workbook027.py",line20,in<module>
test()
File"workbook027.py",line18,intest
test_exception(1)
File"workbook027.py",line10,intest_exception
x=y
NameError:globalname'y'isnotdefined
Notes:
Page161
Ourexcept:clausecaughttheSizeError,butallowedtheNameError
tobeuncaught.
3. WedefineasubclassofofclassException,thenraiseitinaninnerloopand
catchitoutsideofanouterloop:
classBreakException1(Exception):
pass
deftest():
a=[11,22,33,44,55,66,]
b=[111,222,333,444,555,666,]
try:
forxina:
print'outerx:%d'%x
foryinb:
ifx>22andy>444:
raiseBreakException1('leaving
innerloop')
print'innery:%d'%y
print'outerafter'
print''*40
exceptBreakException1,exp:
print'outofloopexp:%s'%exp
test()
Hereiswhatthisprintsoutwhenrun:
outerx:11
innery:111
innery:222
innery:333
innery:444
innery:555
innery:666
outerafter
outerx:22
innery:111
innery:222
innery:333
innery:444
innery:555
innery:666
outerafter
outerx:33
innery:111
innery:222
innery:333
innery:444
outofloopexp:leavinginnerloop
3.6Functions
Afunctionhasthesecharacteristics:
Page162
Itgroupsablockofcodetogethersothatwecancallitbyname.
Itenablesustopassvaluesintothethefunctionwhenwecallit.
Itcanreturnsavalue(evenifNone).
Whenafunctioniscalled,ithasitsownnamespace.Variablesinthefunctionare
localtothefunction(anddisappearwhenthefunctionexits).
Afunctionisdefinedwiththedef:statement.Hereisasimpleexample/template:
deffunction_name(arg1,arg2):
local_var1=arg1+1
local_var2=arg2*2
returnlocal_var1+local_var2
And,hereisanexampleofcallingthisfunction:
result=function_name(1,2)
Hereareafewnotesofexplanation:
Theabovedefinesafunctionwhosenameisfunction_name.
Thefunctionfunction_namehastwoarguments.Thatmeansthatwecanand
mustpassinexactlytwovalueswhenwecallit.
Thisfunctionhastwolocalvariables,local_var1andlocal_var2.These
variablesarelocalinthesensethatafterwecallthisfunction,thesetwovariables
arenotavailableinthelocationofthecaller.
Whenwecallthisfunction,itreturnsonevalue,specificallythesumof
local_var1andlocal_var2.
Exercises:
1. Writeafunctionthattakesalistofintegersasanargument,andreturnsthesum
oftheintegersinthatlist.
Solutions:
1. Thereturnstatementenablesustoreturnavaluefromafunction:
deflist_sum(values):
sum=0
forvalueinvalues:
sum+=value
returnsum
deftest():
a=[11,22,33,44,]
printlist_sum(a)
test()
3.6.1Optionalargumentsanddefaultvalues
Youcanprovideadefaultvalueforanargumenttoafunction.
Ifyoudo,thatargumentisoptional(whenthefunctioniscalled).
Page163
Hereareafewthingstolearnaboutoptionalarguments:
Provideadefaultvaluewithanequalsignandavalue.Example:
defsample_func(arg1,arg2,arg3='empty',arg4=0):
Allparameterswithdefaultvaluesmustbeafter(totherightof)normal
parameters.
Donotuseamutableobjectasadefaultvalue.Becausethedef:statementis
evaluatedonlyonceandnoteachtimethefunctioniscalled,themutableobject
mightbesharedacrossmultiplecallstothefunction.Donotdothis:
defsample_func(arg1,arg2=[]):
Instead,dothis:
defsample_func(arg1,arg2=None):
ifarg2isNone:
arg2=[]
Hereisanexamplethatillustrateshowthismightgowrong:
defadder(a,b=[]):
b.append(a)
returnb
deftest():
printadder('aaa')
printadder('bbb')
printadder('ccc')
test()
Which,whenexecuted,displaysthefollowing:
['aaa']
['aaa','bbb']
['aaa','bbb','ccc']
Exercises:
1. Writeafunctionthatwritesastringtoafile.Thefunctiontakestwoarguments:
(1)afilethatisopenforoutputand(2)astring.Givethesecondargument(the
string)adefaultvaluesothatwhenthesecondargumentisomitted,anempty,
blanklineiswrittentothefile.
2. Writeafunctionthattakesthefollowingarguments:(1)aname,(2)avalue,and
(3)andoptionaldictionary.Thefunctionaddsthevaluetothedictionaryusingthe
nameasakeyinthedictionary.
Solutions:
1. Wecanpassafileaswewouldanyotherobject.And,wecanuseanewline
characterasadefaultparametervalue:
importsys
defwriter(outfile,msg='\n'):
outfile.write(msg)
deftest():
writer(sys.stdout,'aaaaa\n')
Page164
writer(sys.stdout)
writer(sys.stdout,'bbbbb\n')
test()
Whenrunfromthecommandline,thisprintsoutthefollowing:
aaaaa
bbbbb
2. Inthissolutionwearecarefulnottouseamutableobjectasadefaultvalue:
defadd_to_dict(name,value,dic=None):
ifdicisNone:
dic={}
dic[name]=value
returndic
deftest():
dic1={'albert':'cute',}
printadd_to_dict('barry','funny',dic1)
printadd_to_dict('charlene','smart',dic1)
printadd_to_dict('darryl','outrageous')
printadd_to_dict('eddie','friendly')
test()
Ifwerunthisscript,wesee:
{'barry':'funny','albert':'cute'}
{'barry':'funny','albert':'cute','charlene':
'smart'}
{'darryl':'outrageous'}
{'eddie':'friendly'}
Notes:
It'simportantthatthedefaultvalueforthedictionaryisNoneratherthanan
emptydictionary,forexample({}).Rememberthatthedef:statementis
evaluatedonlyonce,whichresultsinasingledictionary,whichwouldbe
sharedbyallcallersthatdonotprovideadictionaryasanargument.
3.6.2Passingfunctionsasarguments
Afunction,likeanyotherobject,canbepassedasanargumenttoafunction.Thisisdue
thethefactthatalmostall(maybeall)objectsinPythonare"firstclassobjects".Afirst
classobjectisonewhichwecan:
1. Storeinadatastructure(e.g.alist,adictionary,...).
2. Passtoafunction.
3. Returnfromafunction.
Exercises:
1. Writeafunctionthattakesthreearguments:(1)aninputfile,(2)anoutputfile,
and(3)afilterfunction:
Argument1isafileopenedforreading.
Page165
Argument2isafileopenedforwriting.
Argument3isafunctionthattakesasingleargument(astring),performsa
transformationonthatstring,andreturnsthetransformedstring.
Theabovefunctionshouldreadeachlineintheinputtextfile,passthatline
throughthefilterfunction,thenwritethat(possibly)transformedlinetothe
outputfile.
Now,writeoneormore"filterfunctions"thatcanbepassedtothefunction
describedabove.
Solutions:
1. Thisscriptaddsorremovescommentcharacterstothelinesofafile:
importsys
deffilter(infile,outfile,filterfunc):
forlineininfile:
line=filterfunc(line)
outfile.write(line)
defadd_comment(line):
line='##%s'%(line,)
returnline
defremove_comment(line):
ifline.startswith('##'):
line=line[3:]
returnline
defmain():
filter(sys.stdin,sys.stdout,add_comment)
main()
Runningthismightproducesomethinglikethefollowing(noteforMSWindows
users:usetypeinsteadofcat):
$cattmp.txt
line1
line2
line3
$cattmp.txt|pythonworkbook005.py
##line1
##line2
##line3
3.6.3Extraargsandkeywordargs
Additionalpositionalargumentspassedtoafunctionthatarenotspecifiedinthefunction
definition(thedef:statement``),arecollectedinanargumentprecededbyasingle
asterisk.Keywordargumentspassedtoafunctionthatarenotspecifiedinthefunction
definitioncanbecollectedinadictionaryandpassedtoanargumentprecededbya
doubleasterisk.
Page166
Examples:
1. Writeafunctionthattakesonepositionalargument,oneargumentwithadefault
value,andalsoextraargsandkeywordargs.
2. Writeafunctionthatpassesallitsarguments,nomatterhowmany,toacallto
anotherfunction.
Solutions:
1. Weuse*argsand**kwargstocollectextraargumentsandextrakeyword
arguments:
defshow_args(x,y=1,*args,**kwargs):
print''*40
print'x:',x
print'y:',y
print'args:',args
deftest():
show_args(1)
show_args(x=2,y=3)
show_args(y=5,x=4)
show_args(4,5,6,7,8)
show_args(11,y=44,a=55,b=66)
test()
Runningthisscriptproducesthefollowing:
x:1
y:1
args:()
kwargs:{}
x:2
y:3
args:()
kwargs:{}
x:4
y:5
args:()
kwargs:{}
x:4
y:5
args:(6,7,8)
kwargs:{}
x:11
y:44
args:()
kwargs:{'a':55,'b':66}
Notes:
Page167
Thespellingofargsandkwargsisnotfixed,butthe
2. Weuseargsandkwargstocatchandpassonallarguments:
deffunc1(*args,**kwargs):
print'args:%s'%(args,)
print'kwargs:%s'%(kwargs,)
deffunc2(*args,**kwargs):
print'before'
func1(*args,**kwargs)
print'after'
deftest():
func2('aaa','bbb','ccc',arg1='ddd',arg2='eee')
test()
Whenwerunthis,itprintsthefollowing:
before
args:('aaa','bbb','ccc')
kwargs:{'arg1':'ddd','arg2':'eee'}
after
Notes:
Inafunctioncall,the*operatorunrollsalistintoindividualpositional
arguments,andthe**operatorunrollsadictionaryintoindividualkeyword
arguments.
3.6.3.1Orderofarguments(positional,extra,andkeywordargs)
Inafunctiondefinition,argumentsmustappearinthefollowingorder,fromlefttoright:
1. Positional(normal,plain)arguments
2. Argumentswithdefaultvalues,ifany
3. Extraargumentsparameter(procededbysingleasterisk),ifpresent
4. Keywordargumentsparameter(procededbydoubleasterisk),ifpresent
Inafunctioncall,argumentsmustappearinthefollowingorder,fromlefttoright:
1. Positional(plain)arguments
2. Extraarguments,ifpresent
3. Keywordarguments,ifpresent
3.6.4Functionsandducktypingandpolymorphism
Iftheargumentsandreturnvalueofafunctionsatisfysomedescription,thenwecansay
thatthefunctionispolymorphicwithrespecttothatdescription.
Ifthesomeofthemethodsofanobjectsatisfysomedescription,thenwecansaythatthe
objectispolymorphicwithrespecttothatdescription.
Basically,whatthisdoesistoenableustouseafunctionoranobjectanywherethat
functionsatisfiestherequirementsgivenbyadescription.
Page168
Exercises:
1. Implementafunctionthattakestwoarguments:afunctionandanobject.It
appliesthefunctionargumenttotheobject.
2. Implementafunctionthattakestwoarguments:alistoffunctionsandanobject.
Itapplieseachfunctioninthelisttotheargument.
Solutions:
1. Wecanpassafunctionasanargumenttoafunction:
deffancy(obj):
print'fancyfancy%sfancyfancy'%(obj,)
defplain(obj):
print'plain%splain'%(obj,)
defshow(func,obj):
func(obj)
defmain():
a={'aa':11,'bb':22,}
show(fancy,a)
show(plain,a)
main()
2. Wecanalsoputfunctions(functionobjects)inadatastructure(forexample,a
list),andthenpassthatdatastructuretoafunction:
deffancy(obj):
print'fancyfancy%sfancyfancy'%(obj,)
defplain(obj):
print'plain%splain'%(obj,)
Func_list=[fancy,plain,]
defshow(funcs,obj):
forfuncinfuncs:
func(obj)
defmain():
a={'aa':11,'bb':22,}
show(Func_list,a)
main()
NoticethatPythonsupportspolymorphism(withor)withoutinheritance.Thistypeof
polymorphismisenabledbywhatiscalledducktyping.Formoreonthissee:Duck
typinghttp://en.wikipedia.org/wiki/Duck_typingatWikipedia.
Page169
3.6.5Recursivefunctions
Arecursivefunctionisafunctionthatcallsitself.
Arecursivefunctionmusthavealimitingcondition,orelseitwillloopendlessly.
Eachrecursivecallconsumesspaceonthefunctioncallstack.Therefore,thenumberof
recursionsmusthavesomereasonableupperbound.
Exercises:
1. Writearecursivefunctionthatprintsinformationabouteachnodeinthe
followingtreestructuredatastructure:
Tree={
'name':'animals',
'left_branch':{
'name':'birds',
'left_branch':{
'name':'seedeaters',
'left_branch':{
'name':'housefinch',
'left_branch':None,
'right_branch':None,
},
'right_branch':{
'name':'whitecrownedsparrow',
'left_branch':None,
},
},
'right_branch':{
'name':'insecteaters',
'left_branch':{
'name':'hermitthrush',
'left_branch':None,
},
'right_branch':{
'name':'blackheadedphoebe',
'left_branch':None,
},
},
},
}
Solutions:
1. Wewritearecursivefunctiontowalkthewholetree.Therecursivefunctioncalls
itselftoprocesseachchildofanodeinthetree:
Tree={
'name':'animals',
'left_branch':{
'name':'birds',
Page170
'left_branch':{
'name':'seedeaters',
'left_branch':{
'name':'housefinch',
'left_branch':None,
},
'right_branch':{
'name':'whitecrownedsparrow',
'left_branch':None,
},
},
'right_branch':{
'name':'insecteaters',
'left_branch':{
'name':'hermitthrush',
'left_branch':None,
},
'right_branch':{
'name':'blackheadedphoebe',
'left_branch':None,
},
},
},
}
Indents=[''*idxforidxinrange(10)]
defwalk_and_show(node,level=0):
ifnodeisNone:
return
print'%sname:%s'%(Indents[level],node['name'],
)
level+=1
walk_and_show(node['left_branch'],level)
walk_and_show(node['right_branch'],level)
deftest():
walk_and_show(Tree)
test()
Notes:
Later,youwilllearnhowtocreateequivalentdatastructuresusingclassesand
OOP(objectorientedprogramming).FormoreonthatseeRecursivecallsto
methodsinthisdocument.
3.6.6Generatorsanditerators
The"iteratorprotocol"defineswhataniteratorobjectmustdoinordertobeusableinan
Page171
"iteratorcontext"suchasaforstatement.Theiteratorprotocolisdescribedinthe
standardlibraryreference:IteratorTypeshttp://docs.python.org/lib/typeiter.html
Aneasywaytodefineanobjectthatobeystheiteratorprotocolistowriteagenerator
function.Ageneratorfunctionisafunctionthatcontainsoneormoreyieldstatements.
Ifafunctioncontainsatleastoneyieldstatement,thenthatfunctionwhencalled,
returnsgeneratoriterator,whichisanobjectthatobeystheiteratorprotocol,i.e.it'san
iteratorobject.
NotethatinrecentversionsofPython,yieldisanexpression.Thisenablestheconsumer
tocommunicatebackwiththeproducer(thegeneratoriterator).Formoreonthis,see
PEP:342CoroutinesviaEnhancedGeneratorshttp://www.python.org/dev/peps/pep
0342/.
Exercises:
1. ImplementageneratorfunctionThegeneratorproducedshouldyieldall
valuesfromalist/iterablethatsatisfyapredicate.Itshouldapplythetransforms
beforereturneachvalue.Thefunctiontakesthesearguments:
1. valuesAlistofvalues.Actually,itcouldbeanyiterable.
2. predicateAfunctionthattakesasingleargument,performsateston
thatvalue,andreturnsTrueorFalse.
3. transforms(optional)Alistoffunctions.Applyeachfunctioninthislist
andreturnstheresultingvalue.So,forexample,ifthefunctioniscalledlike
this:
result=transforms([11,22],p,[f,g])
thentheresultinggeneratormightreturn:
g(f(11))
2. ImplementageneratorfunctionthattakesalistofURLsasitsargumentand
generatesthecontentsofeachWebpage,onebyone(thatis,itproducesa
sequenceofstrings,theHTMLpagecontents).
Solutions:
1. Hereistheimplementationofafunctionwhichcontainsyield,and,therefore,
producesagenerator:
"""
filter_and_transform
filter_and_transform(content,test_func,
transforms=None)
Returnageneratorthatreturnsitemsfromcontent
afterapplying
thefunctionsintransformsiftheitemsatisfies
test_func.
Arguments:
Page172
1.``values`Àlistofvalues
2.``predicate`Àfunctionthattakesasingle
argument,
performsatestonthatvalue,andreturnsTrue
orFalse.
3.``transforms``(optional)Alistoffunctions.
Applyeach
functioninthislistandreturnstheresulting
value.So,
forexample,ifthefunctioniscalledlike
this::
result=filter_and_transforms([11,22],p,[f,
g])
thentheresultinggeneratormightreturn::
g(f(11))
"""
deffilter_and_transform(content,test_func,
transforms=None):
forxincontent:
iftest_func(x):
iftransformsisNone:
yieldx
elifisiterable(transforms):
forfuncintransforms:
x=func(x)
yieldx
else:
yieldtransforms(x)
defisiterable(x):
flag=True
try:
x=iter(x)
exceptTypeError,exp:
flag=False
returnflag
defiseven(n):
returnn%2==0
deff(n):
returnn*2
defg(n):
returnn**2
deftest():
data1=[11,22,33,44,55,66,77,]
forvalinfilter_and_transform(data1,iseven,f):
print'val:%d'%(val,)
Page173
print''*40
forvalinfilter_and_transform(data1,iseven,[f,
g]):
print''*40
forvalinfilter_and_transform(data1,iseven):
test()
Notes:
Becausefunctionfilter_and_transformcontainsyield,when
called,itreturnsaniteratorobject,whichwecanuseinaforstatement.
Thesecondparameteroffunctionfilter_and_transformtakesany
functionwhichtakesasingleargumentandreturnsTrueorFalse.Thisisan
exampleofpolymorphismand"ducktyping"(seeDuckTyping
http://en.wikipedia.org/wiki/Duck_typing).Ananalogousclaimcanbemade
aboutthethirdparameter.
2. Thefollowingfunctionusestheurllibmoduleandtheyieldfunctionto
generatethecontentsofasequenceofWebpages:
importurllib
Urls=[
'http://yahoo.com',
program
]
defwalk(url_list):
forurlinurl_list:
stuff=f.read()
f.close()
yieldstuff
deftest():
forxinwalk(Urls):
print'length:%d'%(len(x),)
test()
WhenIrunthis,Isee:
$pythongenerator_example.py
length:9554
length:16748
length:11487
3.7Objectorientedprogrammingandclasses
ClassesprovidePython'swaytodefinenewdatatypesandtodoOOP(objectoriented
Page174
programming).
Ifyouhavemadeitthisfar,youhavealreadyusedlotsofobjects.Youhavebeena
"consumer"ofobjectsandtheirservices.Now,youwilllearnhowtodefineand
implementnewkindsofobjects.Youwillbecomea"producer"ofobjects.Youwill
definenewclassesandyouwillimplementthecapabilities(methods)ofeachnewclass.
Aclassisdefinedwiththeclassstatement.Thefirstlineofaclassstatementisa
header(ithasacolonattheend),anditspecifiesthenameoftheclassbeingdefinedand
an(optional)superclass.Andthatheaderintroducesacompoundstatement:specifically,
thebodyoftheclassstatementwhichcontainsindented,nestedstatements,
importantly,defstatementsthatdefinethemethodsthatcanbecalledoninstancesofthe
objectsimplementedbythisclass.
Exercises:
1. Defineaclasswithonemethodshow.Thatmethodshouldprintout"Hello".
Then,createaninstanceofyourclass,andcalltheshowmethod.
Solutions:
1. Asimpleinstancemethodcanhavetheselfparameterandnoothers:
classDemo(object):
defshow(self):
print'hello'
deftest():
a=Demo()
a.show()
test()
Notes:
Noticethatweuseobjectasasuperclass,becausewewanttodefinean
"newstyle"classandbecausethereisnootherclassthatwewantasa
superclass.Seethefollowingformoreinformationonnewstyleclasses:
NewstyleClasseshttp://www.python.org/doc/newstyle/.
InPython,wecreateaninstanceofaclassbycallingtheclass,thatis,we
applythefunctioncalloperator(parentheses)totheclass.
3.7.1Theconstructor
Aclasscandefinemethodswithspecialnames.Youhaveseemsomeofthesebefore.
Thesenamesbeginandendwithadoubleunderscore.
Oneimportantspecialnameis__init__.It'stheconstructorforaclass.Itiscalled
eachtimeaninstanceoftheclassiscreated.Implementingthismethodinaclassgivesus
achancetoinitializeeachinstanceofourclass.
Exercises:
1. ImplementaclassnamedPlantthathasaconstructorwhichinitializestwo
Page175
instancevariables:nameandsize.Also,inthisclass,implementamethod
namedshowthatprintsoutthevaluesoftheseinstancevariables.Createseveral
instancesofyourclassand"show"them.
2. ImplementaclassnameNodethathastwoinstancevariables:dataand
children,wheredataisany,arbitraryobjectandchildrenisalistofchild
Nodes.Alsoimplementamethodnamedshowthatrecursivelydisplaysthe
nodesina"tree".Createaninstanceofyourclassthatcontainsseveralchild
instancesofyourclass.Calltheshowmethodontheroot(topmost)objectto
showthetree.
Solutions:
1. Theconstructorforaclassisamethodwiththespecialname__init__:
classPlant(object):
def__init__(self,name,size):
self.name=name
self.size=size
defshow(self):
print'name:"%s"size:%d'%(self.name,
self.size,)
deftest():
p1=Plant('Eggplant',25)
p2=Plant('Tomato',36)
plants=[p1,p2,]
forplantinplants:
plant.show()
test()
Notes:
Ourconstructortakestwoarguments:nameandsize.Itsavesthosetwo
valuesasinstancevariables,thatisinattributesoftheinstance.
Theshow()methodprintsoutthevalueofthosetwoinstancevariables.
2. Itisagoodideatoinitializeallinstancevariablesintheconstructor.Thatenables
someonereadingourcodetolearnaboutalltheinstancevariablesofaclassby
lookinginasinglelocation:
#simple_node.py
Indents=[''*nforninrange(10)]
classNode(object):
def__init__(self,name=None,children=None):
self.name=name
ifchildrenisNone:
self.children=[]
else:
defshow_name(self,indent):
print'%sname:"%s"'%(Indents[indent],
self.name,)
defshow(self,indent=0):
Page176
self.show_name(indent)
indent+=1
child.show(indent)
deftest():
n1=Node('N1')
n2=Node('N2')
n3=Node('N3')
n4=Node('N4')
n5=Node('N5',[n1,n2,])
n6=Node('N6',[n3,n4,])
n7=Node('N7',[n5,n6,])
n7.show()
test()
Notes:
Noticethatwedonotusetheconstructorforalist([])asadefaultvaluefor
thechildrenparameteroftheconstructor.Alistismutableandwouldbe
createdonlyonce(whentheclassstatementisexecuted)andwouldbeshared.
3.7.2InheritanceImplementingasubclass
Asubclassextendsorspecializesasuperclassbyaddingadditionalmethodstothe
superclassandbyoverridingmethods(withthesamename)thatalreadyexistinthe
superclass.
Exercises:
1. ExtendyourNodeexerciseabovebyaddingtwoadditionalsubclassesofthe
Nodeclass,onenamedPlantandtheothernamedAnimal.ThePlantclass
alsohasaheightinstancevariableandtheAnimalclassalsohasacolor
instancevariable.
Solutions:
1. WecanimportourpreviousNodescript,thenimplementclassesthathavethe
Nodeclassasasuperclass:
fromsimple_nodeimportNode,Indents
classPlant(Node):
def__init__(self,name,height=1,children=None):
Node.__init__(self,name,children)
self.height=height
print'%sheight:%s'%(Indents[indent],
self.height,)
indent+=1
child.show(indent)
Page177
classAnimal(Node):
def__init__(self,name,color='nocolor',
children=None):
Node.__init__(self,name,children)
self.color=color
print'%scolor:"%s"'%(Indents[indent],
self.color,)
indent+=1
child.show(indent)
deftest():
n1=Animal('scrubjay','grayblue')
n2=Animal('raven','black')
n3=Animal('americankestrel','brown')
n4=Animal('redshoulderedhawk','brownand
gray')
n5=Animal('corvid','none',[n1,n2,])
n6=Animal('raptor',children=[n3,n4,])
n7a=Animal('bird',children=[n5,n6,])
n1=Plant('valleyoak',50)
n2=Plant('canyonliveoak',40)
n3=Plant('jefferypine',120)
n4=Plant('ponderosapine',140)
n5=Plant('oak',children=[n1,n2,])
n6=Plant('conifer',children=[n3,n4,])
n7b=Plant('tree',children=[n5,n6,])
n8=Node('birdsandtrees',[n7a,n7b,])
n8.show()
test()
Notes:
TheshowmethodinclassPlantcallstheshow_namemethodinits
superclassusingself.show_name(...).Pythonsearchesupthe
inheritancetreetofindtheshow_namemethodinclassNode.
Theconstructor(__init__)inclassesPlantandAnimaleachcallthe
constructorinthesuperclassbyusingthenameofthesuperclass.Whythe
difference?Because,if(inthePlantclass,forexample)itused
self.__init__(...)itwouldbecallingthe__init__inthePlant
class,itself.So,itbypassesitselfbyreferencingtheconstructorinthe
superclassdirectly.
Thisexercisealsodemonstrates"polymorphism"Theshowmethodis
calledanumberoftimes,butwhichimplementationexecutesdependson
whichinstanceitiscalledon.Callingontheshowmethodonaninstanceof
classPlantresultsinacalltoPlant.show.Callingtheshowmethodon
aninstanceofclassAnimalresultsinacalltoAnimal.show.Andsoon.It
isimportantthateachshowmethodtakesthecorrectnumberofarguments.
Page178
3.7.3Classesandpolymorphism
Pythonalsosupportsclassbasedpolymorphism,whichwas,bytheway,demonstratedin
thepreviousexample.
Exercises:
1. Writethreeclasses,eachofwhichimplementashow()methodthattakesone
argument,astring.Theshowmethodshouldprintoutthenameoftheclassand
themessage.Thencreatealistofinstancesandcalltheshow()methodoneach
objectinthelist.
Solution:
1. Weimplementthreesimpleclassesandthencreatealistofinstancesofthese
classes:
classA(object):
defshow(self,msg):
print'classAmsg:"%s"'%(msg,)
classB(object):
defshow(self,msg):
print'classBmsg:"%s"'%(msg,)
classC(object):
defshow(self,msg):
print'classCmsg:"%s"'%(msg,)
deftest():
objs=[A(),B(),C(),A(),]
foridx,objinenumerate(objs):
msg='message#%d'%(idx+1,)
obj.show(msg)
test()
Notes:
Wecancalltheshow()methodinanyobjectinthelistobjsaslongaswe
passinasingleparameter,thatis,aslongasweobeytherequirementsof
ducktyping.Wecandothisbecauseallobjectsinthatlistimplementa
show()method.
Inastaticallytypedlanguage,thatisalanguagewherethetypeis(also)
presentinthevariable,alltheinstancesinexamplewouldhavetodescend
fromacommonsuperclassandthatsuperclasswouldhavetoimplementa
show()method.Pythondoesnotimposethisrestriction.And,because
variablesarenotnottypedinPython,perhapsthatwouldnotevenpossible.
Noticethatthisexampleofpolymorphismworkseventhoughthesethree
classes(A,B,andC)arenotrelated(forexample,inaclasshierarchy).All
thatisrequiredforpolymorphismtoworkinPythonisforthemethodnames
tobethesameandtheargumentstobecompatible.
Page179
3.7.4Recursivecallstomethods
Amethodinaclasscanrecusivelycallitself.Thisisverysimilartothewayinwhichwe
implementedrecursivefunctionssee:Recursivefunctions.
Exercises:
1. ReimplementthebinarytreeofanimalsandbirdsdescribedinRecursive
functions,butthistime,useaclasstorepresenteachnodeinthetree.
2. Solvethesameproblem,butthistimeimplementatreeinwhicheachnodecan
haveanynumberofchildren(ratherthanexactly2children).
Solutions:
1. Weimplementaclasswiththreeinstancevariables:(1)name,(2)leftbranch,and
(3)rightbranch.Then,weimplementashow()methodthatdisplaysthename
andcallsitselftoshowthechildrenineachsubtree:
Indents=[''*idxforidxinrange(10)]
classAnimalNode(object):
def__init__(self,name,left_branch=None,
right_branch=None):
self.name=name
self.left_branch=left_branch
self.right_branch=right_branch
defshow(self,level=0):
print'%sname:%s'%(Indents[level],
self.name,)
level+=1
ifself.left_branchisnotNone:
self.left_branch.show(level)
ifself.right_branchisnotNone:
self.right_branch.show(level)
Tree=AnimalNode('animals',
AnimalNode('birds',
AnimalNode('seedeaters',
AnimalNode('housefinch'),
AnimalNode('whitecrownedsparrow'),
),
AnimalNode('insecteaters',
AnimalNode('hermitthrush'),
AnimalNode('blackheadedphoebe'),
),
),
None,
)
deftest():
Tree.show()
Page180
test()
2. Insteadofusingaleftbranchandarightbranch,inthissolutionweusealistto
representthechildrenofanode:
classAnimalNode(object):
def__init__(self,data,children=None):
self.data=data
ifchildrenisNone:
self.children=[]
else:
defshow(self,level=''):
print'%sdata:%s'%(level,self.data,)
level+=''
child.show(level)
Tree=AnimalNode('animals',[
AnimalNode('birds',[
AnimalNode('seedeaters',[
AnimalNode('housefinch'),
AnimalNode('whitecrownedsparrow'),
AnimalNode('lessergoldfinch'),
]),
AnimalNode('insecteaters',[
AnimalNode('hermitthrush'),
AnimalNode('blackheadedphoebe'),
]),
])
])
deftest():
Tree.show()
test()
Notes:
Werepresentthechildrenofanodeasalist.Eachnode"hasa"listof
children.
Noticethatbecausealistismutable,wedonotusealistconstructor([])in
theinitializerofthemethodheader.Instead,weuseNone,thenconstructan
emptylistinthebodyofthemethodifnecessary.SeesectionOptional
argumentsanddefaultvaluesformoreonthis.
We(recursively)calltheshowmethodforeachnodeinthechildrenlist.
Sinceanodewhichhasnochildren(aleafnode)willhaveanempty
childrenlist,thisprovidesalimitconditionforourrecursion.
3.7.5Classvariables,classmethods,andstaticmethods
Aclassvariableisonewhosesinglevalueissharedbyallinstancesoftheclassand,in
fact,issharedbyallwhohaveaccesstotheclass(object).
Page181
"Normal"methodsareinstancemethods.Aninstancemethodreceivestheinstanceasits
firstargument.Ainstancemethodisdefinedbyusingthedefstatementinthebodyofa
classstatement.
Aclassmethodreceivestheclassasitsfirstargument.Aclassmethodisdefinedby
defininganormal/instancemethod,thenusingtheclassmethodbuiltinfunction.For
example:
classASimpleClass(object):
description='asimpleclass'
defshow_class(cls,msg):
print'%s:%s'%(cls.description,msg,)
show_class=classmethod(show_class)
Astaticmethoddoesnotreceiveanythingspecialasitsfirstargument.Astaticmethodis
definedbydefininganormal/instancemethod,thenusingthestaticmethodbuiltin
function.Forexample:
classASimpleClass(object):
description='asimpleclass'
defshow_class(msg):
print'%s:%s'%(ASimpleClass.description,msg,)
show_class=staticmethod(show_class)
Ineffect,bothclassmethodsandstaticmethodsaredefinedbycreatinganormal
(instance)method,thencreatingawrapperobject(aclassmethodorstaticmethod)using
theclassmethodorstaticmethodbuiltinfunction.
Exercises:
1. Implementaclassthatkeepsarunningtotalofthenumberofinstancescreated.
2. Implementanothersolutiontothesameproblem(aclassthatkeepsarunning
totalofthenumberofinstances),butthistimeuseastaticmethodinsteadofa
classmethod.
Solutions:
1. Weuseaclassvariablenamedinstance_count,ratherthananinstance
variable,tokeeparunningtotalofinstances.Then,weincrementthatvariable
eachtimeaninstanceiscreated:
classCountInstances(object):
instance_count=0
def__init__(self,name='noname'):
self.name=name
CountInstances.instance_count+=1
defshow(self):
print'name:"%s"'%(self.name,)
defshow_instance_count(cls):
print'instancecount:%d'%
Page182
(cls.instance_count,)
show_instance_count=
classmethod(show_instance_count)
deftest():
instances=[]
instances.append(CountInstances('apple'))
instances.append(CountInstances('banana'))
instances.append(CountInstances('cherry'))
instances.append(CountInstances())
forinstanceininstances:
instance.show()
CountInstances.show_instance_count()
test()
Notes:
Whenwerunthisscript,itprintsoutthefollowing:
name:"apple"
name:"banana"
name:"cherry"
name:"noname"
instancecount:4
Thecalltotheclassmethodbuiltinfunctioneffectivelywrapsthe
show_instance_countmethodinaclassmethod,thatis,inamethod
thattakesaclassobjectasitsfirstargumentratherthananinstanceobject.To
readmoreaboutclassmethod,gotoBuiltinFunctions
http://docs.python.org/lib/builtinfuncs.htmlandsearchfor"classmethod".
2. Astaticmethodtakesneitheraninstance(self)noraclassasitsfirst
paramenter.And,staticmethodiscreatedwiththestaticmethod()builtin
function(ratherthanwiththeclassmethod()builtin):
instance_count=0
self.name=name
defshow(self):
defshow_instance_count():
print'instancecount:%d'%(
CountInstances.instance_count,)
show_instance_count=
staticmethod(show_instance_count)
deftest():
instances=[]
Page183
instance.show()
test()
3.7.5.1Decoratorsforclassmethodandstaticmethod
Adecoratorenablesustodowhatwedidinthepreviousexamplewithasomewhat
simplersyntax.
Forsimplecases,thedecoratorsyntaxenablesustodothis:
@functionwrapper
defmethod1(self):
o
o
o
insteadofthis:
defmethod1(self):
o
o
o
method1=functionwrapper(method1)
So,wecanwritethis:
@classmethod
defmethod1(self):
o
o
o
insteadofthis:
defmethod1(self):
o
o
o
method1=classmethod(method1)
Exercises:
1. ImplementtheCountInstancesexampleabove,butuseadecoratorrather
thantheexplicitcalltoclassmethod.
Solutions:
1. Adecoratorisaneasierandcleanerwaytodefineaclassmethod(orastatic
Page184
method):
instance_count=0
self.name=name
defshow(self):
@classmethod
defshow_instance_count(cls):
print'instancecount:%d'%
(cls.instance_count,)
#Notethatthefollowinglinehasbeenreplacedby
#theclassmethoddecorator,above.
#show_instance_count=
classmethod(show_instance_count)
deftest():
instances=[]
instance.show()
test()
3.8AdditionalandAdvancedTopics
3.8.1Decoratorsandhowtoimplementthem
Decoratorscanbeusedto"wrap"afunctionwithanotherfunction.
Whenimplementingadecorator,itishelpfultorememberthatthefollowingdecorator
application:
@dec
deffunc(arg1,arg2):
pass
isequivalentto:
deffunc(arg1,arg2):
pass
func=dec(func)
Page185
Therefore,toimplementadecorator,wewriteafunctionthatreturnsafunctionobject,
sincewereplacethevalueoriginallyboundtothefunctionwiththisnewfunctionobject.
Itmaybehelpfultotaketheviewthatwearecreatingafunctionthatisawrapperforthe
originalfunction.
Exercises:
1. Writeadecoratorthatwritesamessagebeforeandafterexecutingafunction.
Solutions:
1. Afunctionthatcontainsandreturnsaninnerfunctioncanbeusedtowrapa
function:
deftrace(func):
definner(*args,**kwargs):
print'>>'
func(*args,**kwargs)
print'<<'
returninner
@trace
deffunc1(x,y):
print'x:',x,'y:',y
func2((x,y))
@trace
deffunc2(content):
print'content:',content
deftest():
func1('aa','bb')
test()
Notes:
Yourinnerfunctioncanuse*argsand**kwargstoenableittocall
functionswithanynumberofarguments.
3.8.1.1Decoratorswitharguments
Decoratorscanalsotakearguments.
Thefollowingdecoratorwitharguments:
@dec(argA,argB)
deffunc(arg1,arg2):
pass
isequivalentto:
deffunc(arg1,arg2):
pass
func=dec(argA,argB)(func)
Becausethedecorator'sargumentsarepassedtotheresultofcallingthedecoratoronthe
Page186
decoratedfunction,youmayfinditusefultoimplementadecoratorwithargumentsusing
afunctioninsideafunctioninsideafunction.
Exercises:
1. Writeandtestadecoratorthattakesoneargument.Thedecoratorprintsa
messagealongwiththevalueoftheargumentbeforeandafterenteringthe
decoratedfunction.
Solutions:
1. Implementthisdecoratorthattakesargumentswithafunctioncontaininganested
functionwhichinturncontainsanestedfunction:
deftrace(msg):
definner1(func):
definner2(*args,**kwargs):
print'>>[%s]'%(msg,)
retval=func(*args,**kwargs)
print'<<[%s]'%(msg,)
returnretval
returninner2
returninner1
@trace('tracingfunc1')
deffunc1(x,y):
print'x:',x,'y:',y
result=func2((x,y))
returnresult
deffunc2(content):
returncontent*3
deftest():
result=func1('aa','bb')
test()
3.8.1.2Stackeddecorators
Decoratorscanbe"stacked".
Thefollowingstackeddecorators:
@dec2
@dec1
deffunc(arg1,arg2,...):
pass
areequivalentto:
deffunc(arg1,arg2,...):
pass
Page187
func=dec2(dec1(func))
Exercises:
1. Implementadecorator(asabove)thattracescallstoadecoratedfunction.Then
"stack"thatwithanotherdecoratorthatprintsahorizontallineofdashesbefore
andaftercallingthefunction.
2. Modifyyoursolutiontotheaboveexercisesothatthedecoratorthatprintsthe
horizontallinetakesoneargument:acharacter(orcharacters)thatcanberepeated
toproduceahorizontalline/separator.
Solutions:
1. Reuseyourtracingfunctionfromthepreviousexercise,thenwriteasimple
decoratorthatprintsarowofdashes:
deftrace(msg):
definner1(func):
returnretval
returninner2
returninner1
defhorizontal_line(func):
definner(*args,**kwargs):
print''*50
print''*50
returnretval
returninner
deffunc1(x,y):
print'x:',x,'y:',y
result=func2((x,y))
returnresult
@horizontal_line
deffunc2(content):
returncontent*3
deftest():
test()
2. Onceagain,adecoratorwithargumentscanbeimplementedwithafunction
nestedinsideafunctionwhichisnestedinsideafunction.Thisremainsthesame
whetherthedecoratorisusedasastackeddecoratorornot.Hereisasolution:
Page188
deftrace(msg):
definner1(func):
returnretval
returninner2
returninner1
defhorizontal_line(line_chr):
definner1(func):
printline_chr*15
printline_chr*15
returnretval
returninner2
returninner1
deffunc1(x,y):
print'x:',x,'y:',y
result=func2((x,y))
returnresult
@horizontal_line('<**>')
deffunc2(content):
returncontent*3
deftest():
test()
3.8.1.3Morehelpwithdecorators
Thereismoreaboutdecoratorshere:
Pythonsyntaxandsemantics
http://en.wikipedia.org/wiki/Python_syntax_and_semantics#Decoratorsat
Wikipedia.
PythonDecoratorLibraryhttp://wiki.python.org/moin/PythonDecoratorLibrary
atthePythonWikihaslotsofsamplecode.
Kent'sKornerPythonDecorators
http://personalpages.tds.net/~kent37/kk/00001.htmlhashelpfulexplanationsand
towardtheendgivesreferencestoothersourcesofinformationondecorators.
PEP318DecoratorsforFunctionsandMethods
http://www.python.org/dev/peps/pep0318/istheformalproposaland
Page189
specificationforPythondecorators.
3.8.2Iterables
3.8.2.1AfewpreliminariesonIterables
Definition:iterable(adjective)thatwhichcanbeiteratedover.
Agoodtestofwhethersomethingisiterableiswhetheritcanbeusedinafor:
statement.Forexample,ifwecanwriteforiteminX:,thenXisiterable.Hereis
anothersimpletest:
defisiterable(x):
try:
y=iter(x)
exceptTypeError,exp:
returnFalse
returnTrue
Somekindsofiterables:
ContainersWecaniterateoverlists,tuples,dictionaries,sets,strings,andother
containers.
Somebuiltin(noncontainer)typesExamples:
Atextfileopeninreadmodeisiterable:ititeratesoverthelinesinthefile.
ThexrangetypeSeeXRangeTypehttp://docs.python.org/lib/typesseq
xrange.html.It'susefulwhenyouwantalargesequenceofintegerstoiterate
over.
Instancesofclassesthatobeytheiteratorprotocol.Foradescriptionoftheiterator
protocol,seeIteratorTypeshttp://docs.python.org/lib/typeiter.html.Hint:Type
dir(obj)andlookfor"__iter__"and"next".
GeneratorsAnobjectreturnedbyanyfunctionormethodthatcontainsyield.
Exercises:
1. Implementaclasswhoseinstancesareinterable.Theconstructortakesalistof
URLsasitsargument.Aninstanceofthisclass,wheniteratedover,generatesthe
contentoftheWebpageatthataddress.
Solutions:
1. Weimplementaclassthathas__iter__()andnext()methods:
importurllib
classWebPages(object):
def__init__(self,urls):
self.urls=urls
self.current_index=0
def__iter__(self):
self.current_index=0
returnself
Page190
defnext(self):
ifself.current_index>=len(self.urls):
raiseStopIteration
url=self.urls[self.current_index]
self.current_index+=1
content=f.read()
f.close()
returncontent
deftest():
urls=[
'http://www.python.org',
'http://en.wikipedia.org/',
'http://en.wikipedia.org/wiki/Python_(programming_langu
age)',
]
pages=WebPages(urls)
forpageinpages:
print'length:%d'%(len(page),)
pages=WebPages(urls)
print''*50
page=pages.next()
page=pages.next()
page=pages.next()
page=pages.next()
test()
3.8.2.2Morehelpwithiterables
TheitertoolsmoduleinthePythonstandardlibraryhashelpersforiterators:
http://docs.python.org/library/itertools.html#moduleitertools
3.9ApplicationsandRecipes
3.9.1XMLSAX,minidom,ElementTree,Lxml
Exercises:
1. SAXParseanXMLdocumentwithSAX,thenshowsomeinformation(tag,
attributes,characterdata)foreachelement.
2. MinidomParseanXMLdocumentwithminidom,thenwalktheDOMtree
andshowsomeinformation(tag,attributes,characterdata)foreachelement.
HereisasampleXMLdocumentthatyoucanuseforinput:
<?xmlversion="1.0"?>
Page191
<people>
<personid="1"value="abcd"ratio="3.2">
<name>Alberta</name>
<interest>gardening</interest>
<interest>reading</interest>
<category>5</category>
</person>
<personid="2">
<name>Bernardo</name>
<interest>programming</interest>
<category></category>
<agent>
<firstname>Darren</firstname>
<lastname>Diddly</lastname>
</agent>
</person>
<personid="3"value="efgh">
<name>Charlie</name>
<interest>people</interest>
<interest>cats</interest>
<interest>dogs</interest>
<promoter>
<firstname>David</firstname>
<lastname>Donaldson</lastname>
<client>
<fullname>ArnoldApplebee</fullname>
<refid>10001</refid>
</client>
</promoter>
<promoter>
<firstname>Edward</firstname>
<lastname>Eddleberry</lastname>
<client>
<fullname>ArnoldApplebee</fullname>
<refid>10001</refid>
</client>
</promoter>
</person>
</people>
3. ElementTreeParseanXMLdocumentwithElementTree,thenwalktheDOM
treeandshowsomeinformation(tag,attributes,characterdata)foreachelement.
4. lxmlParseanXMLdocumentwithlxml,thenwalktheDOMtreeandshow
someinformation(tag,attributes,characterdata)foreachelement.
5. ModifydocumentwithElementTreeUseElementTreetoreadadocument,then
modifythetree.Showthecontentsofthetree,andthenwriteoutthemodified
document.
6. XPathlxmlsupportsXPath.UsetheXPathsupportinlxmltoaddresseachof
thefollowingintheaboveXMLinstancedocument:
Thetextinallthenameelements
Thevaluesofalltheidattributes
Solutions:
Page192
1. WecanusetheSAXsupportinthePythonstandardlibrary:
"""
ParseandXMLwithSAX.Displayinfoabouteach
element.
Usage:
pythontest_sax.pyinfilename
Examples:
pythontest_sax.pypeople.xml
"""
importsys
fromxml.saximportmake_parser,handler
classTestHandler(handler.ContentHandler):
def__init__(self):
self.level=0
defshow_with_level(self,value):
print'%s%s'%(''*self.level,value,)
defstartDocument(self):
self.show_with_level('Documentstart')
self.level+=1
defendDocument(self):
self.level=1
self.show_with_level('Documentend')
defstartElement(self,name,attrs):
self.show_with_level('startelementname:
"%s"'%(name,))
self.level+=1
defendElement(self,name):
self.level=1
self.show_with_level('endelementname:
"%s"'%(name,))
defcharacters(self,content):
content=content.strip()
ifcontent:
self.show_with_level('characters:"%s"'%
(content,))
parser=make_parser()
handler=TestHandler()
parser.setContentHandler(handler)
parser.parse(infilename)
defusage():
print__doc__
Page193
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)!=1:
usage()
infilename=args[0]
test(infilename)
main()
2. Theminidommodulecontainsaparse()functionthatenablesustoreadan
XMLdocumentandcreateaDOMtree:
"""ProcessanXMLdocumentwithminidom.
Showthedocumenttree.
Usage:
pythonminidom_walk.py[options]infilename
"""
importsys
fromxml.domimportminidom
defshow_tree(doc):
root=doc.documentElement
show_node(root,0)
defshow_node(node,level):
count=0
ifnode.nodeType==minidom.Node.ELEMENT_NODE:
show_level(level)
print'tag:%s'%(node.nodeName,)
forkeyinnode.attributes.keys():
attr=node.attributes.get(key)
show_level(level+1)
print'attributename:%svalue:"%s"'%
(attr.name,
attr.value,)
if(len(node.childNodes)==1and
node.childNodes[0].nodeType==
minidom.Node.TEXT_NODE):
show_level(level+1)
print'data:"%s"'%
(node.childNodes[0].data,)
forchildinnode.childNodes:
count+=1
show_node(child,level+1)
returncount
defshow_level(level):
forxinrange(level):
print'',
Page194
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print__doc__
sys.exit(1)
docname=args[0]
doc=minidom.parse(docname)
show_tree(doc)
test()
3. ElementTreeenablesustoparseanXMLdocumentandcreateaDOMtree:
"""ProcessanXMLdocumentwithelementtree.
Usage:
pythonelementtree_walk.py[options]infilename
"""
importsys
fromxml.etreeimportElementTreeasetree
defshow_tree(doc):
root=doc.getroot()
show_node(root,0)
show_level(level)
print'tag:%s'%(node.tag,)
forkey,valueinnode.attrib.iteritems():
show_level(level+1)
(key,value,)
ifnode.text:
text=node.text.strip()
show_level(level+1)
print'text:"%s"'%(node.text,)
ifnode.tail:
tail=node.tail.strip()
show_level(level+1)
print'tail:"%s"'%(tail,)
forchildinnode.getchildren():
forxinrange(level):
print'',
deftest():
args=sys.argv[1:]
Page195
iflen(args)!=1:
print__doc__
sys.exit(1)
docname=args[0]
doc=etree.parse(docname)
show_tree(doc)
test()
4. lxmlenablesustoparseanXMLdocumentandcreateaDOMtree.Infact,since
lxmlattemptstomimictheElementTreeAPI,ourcodeisverysimilartothatin
thesolutiontotheElementTreeexercise:
Usage:
pythonlxml_walk.py[options]infilename
"""
#
#Imports:
importsys
fromlxmlimportetree
defshow_tree(doc):
root=doc.getroot()
show_node(root,0)
show_level(level)
show_level(level+1)
(key,value,)
ifnode.text:
show_level(level+1)
ifnode.tail:
show_level(level+1)
forxinrange(level):
print'',
Page196
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print__doc__
sys.exit(1)
docname=args[0]
doc=etree.parse(docname)
show_tree(doc)
test()
5. WecanmodifytheDOMtreeandwriteitouttoanewfile:
Modifythedocumenttreeandthenshowitagain.
WritethemodifiedXMLtreetoanewfile.
Usage:
pythonelementtree_walk.py[options]infilename
outfilename
Options:
Example:
pythonelementtree_walk.pymyxmldoc.xml
myotherxmldoc.xml
"""
importsys
importos
importgetopt
importtime
#UseElementTree.
fromxml.etreeimportElementTreeasetree
#OruncommenttouseLxml.
#fromlxmlimportetree
defshow_tree(doc):
root=doc.getroot()
show_node(root,0)
show_level(level)
show_level(level+1)
(key,value,)
ifnode.text:
show_level(level+1)
Page197
ifnode.tail:
show_level(level+1)
forxinrange(level):
print'',
defmodify_tree(doc,tag,attrname,attrvalue):
root=doc.getroot()
modify_node(root,tag,attrname,attrvalue)
defmodify_node(node,tag,attrname,attrvalue):
ifnode.tag==tag:
node.attrib[attrname]=attrvalue
modify_node(child,tag,attrname,attrvalue)
deftest(indocname,outdocname):
doc=etree.parse(indocname)
show_tree(doc)
print''*50
date=time.ctime()
modify_tree(doc,'person','date',date)
show_tree(doc)
write_output=False
ifos.path.exists(outdocname):
response=raw_input('Outputfile(%s)exists.
Overwrite?(y/n):'%
outdocname)
ifresponse=='y':
write_output=True
else:
write_output=True
ifwrite_output:
doc.write(outdocname)
print'WrotemodifiedXMLtreeto%s'%
outdocname
else:
print'Didnotwriteoutputfile.'
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help',
])
except:
usage()
Page198
foropt,valinopts:
usage()
iflen(args)!=2:
usage()
indocname=args[0]
outdocname=args[1]
test(indocname,outdocname)
main()
Notes:
TheabovesolutioncontainsanimportstatementforElementTreeand
anotherforlxml.Theoneforlxmliscommentedout,butyoucouldchange
thatifyouwishtouselxmlinsteadofElementTree.Thissolutionwillwork
thesamewaywitheitherElementTreeorlxml.
6. WhenweparseandXMLdocumentwithlxml,eachelement(node)hasan
xpath()method.
#test_xpath.py
fromlxmlimportetree
deftest():
doc=etree.parse('people.xml')
root=doc.getroot()
printroot.xpath("//name/text()")
printroot.xpath("//@id")
test()
And,whenweruntheabovecode,hereiswhatwesee:
$pythontest_xpath.py
['Alberta','Bernardo','Charlie']
['1','2','3']
FormoreonXPathsee:XMLPathLanguage(XPath)
http://www.w3.org/TR/xpath
3.9.2Relationaldatabaseaccess
YoucanfindinformationaboutdatabaseprogramminginPythonhere:Database
Programminghttp://wiki.python.org/moin/DatabaseProgramming/.
FordatabaseaccessweusethePythonDatabaseAPI.Youcanfindinformationaboutit
here:PythonDatabaseAPISpecificationv2.0http://www.python.org/dev/peps/pep
0249/.
TousethedatabaseAPIwedothefollowing:
1. Usethedatabaseinterfacemoduletocreateaconnectionobject.
2. Usetheconnectionobjecttocreateacursorobject.
Page199
3. UsethecursorobjecttoexecuteanSQLquery.
4. Retrieverowsfromthecursorobject,ifneeded.
5. Optionally,commitresultstothedatabase.
6. Closetheconnectionobject.
Ourexamplesusethegadflydatabase,whichiswritteninPython.Ifyouwanttouse
gadfly,youcanfindithere:http://gadfly.sourceforge.net/.gadflyisareasonable
choiceifyouwantaneasytousedatabaseonyourlocalmachine.
Anotherreasonablechoiceforalocaldatabaseissqlite3,whichisinthePython
standardlibrary.HereisadescriptivequotefromtheSQLiteWebsite:
"SQLiteisasoftwarelibrarythatimplementsaselfcontained,
serverless,zeroconfiguration,transactionalSQLdatabaseengine.
SQLiteisthemostwidelydeployedSQLdatabaseengineintheworld.
ThesourcecodeforSQLiteisinthepublicdomain."
Youcanlearnaboutithere:
sqlite3DBAPI2.0interfaceforSQLitedatabases
http://docs.python.org/library/sqlite3.html
SQLitehomepagehttp://www.sqlite.org/
Thepysqlitewebpagehttp://oss.itsystementwicklung.de/trac/pysqlite/
Ifyouwantorneedtouseanother,enterpriseclassdatabase,forexamplePostgreSQL,
MySQL,Oracle,etc.,youwillneedaninterfacemoduleforyourspecificdatabase.You
canfindinformationaboutdatabaseinterfacemoduleshere:Databaseinterfaces
http://wiki.python.org/moin/DatabaseInterfaces
Excercises:
1. Writeascriptthatretrievesalltherowsinatableandprintseachrow.
2. Writeascriptthatretrievesalltherowsinatable,thenusesthecursorasan
iteratortoprinteachrow.
3. Writeascriptthatusesthecursor'sdescriptionattributetoprintoutthename
andvalueofeachfieldineachrow.
4. Writeascriptthatperformsseveraloftheabovetasks,butusessqlite3instead
ofgadfly.
Solutions:
1. WecanexecuteaSQLqueryandthenretrievealltherowswith
fetchall():
importgadfly
deftest():
connection=gadfly.connect("dbtest1",
"plantsdbdir")
cur=connection.cursor()
cur.execute('select*fromplantsdborderby
p_name')
rows=cur.fetchall()
Page200
forrowinrows:
print'2.row:',row
connection.close()
test()
2. Thecursoritselfisaniterator.Ititeratesovertherowsreturnedbyaquery.So,
weexecuteaSQLqueryandthenweusethecursorinafor:statement:
importgadfly
deftest():
connection=gadfly.connect("dbtest1",
"plantsdbdir")
cur=connection.cursor()
p_name')
forrowincur:
printrow
connection.close()
test()
3. Thedescriptionattributeinthecursorisacontainerthathasanitemdescribing
eachfield:
importgadfly
deftest():
p_name')
forfieldincur.description:
print'field:',field
rows=cur.fetchall()
forrowinrows:
foridx,fieldinenumerate(row):
content='%s:"%s"'%
(cur.description[idx][0],field,)
printcontent,
print
connection.close()
test()
Notes:
ThecommaattheendoftheprintstatementtellsPythonnottoprintanew
line.
Thecur.descriptionisasequencecontaininganitemforeachfield.
Afterthequery,wecanextractadescriptionofeachfield.
4. Thesolutionsusingsqlite3areverysimilartothoseusinggadfly.For
informationonsqlite3,see:sqlite3DBAPI2.0interfaceforSQLite
databaseshttp://docs.python.org/library/sqlite3.html#modulesqlite3.
"""
Performoperationsonsqlite3(plants)database.
Page201
Usage:
pythonpy_db_api.pycommand[arg1,...]
Commands:
createcreatenewdatabase.
showshowcontentsofdatabase.
addaddrowtodatabase.Requires3args(name,
descrip,rating).
deleteremoverowfromdatabase.Requires1arg
(name).
Examples:
pythontest1.pycreate
pythontest1.pyshow
pythontest1.pyaddcrenshaw"Themostsucculent
melon"10
pythontest1.pydeletelemon
"""
importsys
importsqlite3
Values=[
('lemon','brightandyellow','7'),
('peach','succulent','9'),
('banana','smoothandcreamy','8'),
('nectarine','tangyandtasty','9'),
('orange','sweetandtangy','8'),
]
Field_defs=[
'p_namevarchar',
'p_descripvarchar',
#'p_ratinginteger',
'p_ratingvarchar',
]
defcreatedb():
connection=sqlite3.connect('sqlite3plantsdb')
cursor=connection.cursor()
q1="createtableplantsdb(%s)"%(',
'.join(Field_defs))
print'createq1:%s'%q1
cursor.execute(q1)
q1="createindexindex1onplantsdb(p_name)"
cursor.execute(q1)
q1="insertintoplantsdb(p_name,p_descrip,
p_rating)values('%s','%s',%s)"
forspecinValues:
q2=q1%spec
print'q2:"%s"'%q2
cursor.execute(q2)
connection.commit()
showdb1(cursor)
connection.close()
Page202
defshowdb():
connection,cursor=opendb()
showdb1(cursor)
connection.close()
defshowdb1(cursor):
cursor.execute("select*fromplantsdborderby
p_name")
hr()
description=cursor.description
printdescription
print'description:'
forrowdescriptionindescription:
print'%s'%(rowdescription,)
hr()
rows=cursor.fetchall()
printrows
print'rows:'
forrowinrows:
print'%s'%(row,)
hr()
print'content:'
forrowinrows:
descrip=row[1]
name=row[0]
rating='%s'%row[2]
print'%s%s%s'%(
name.ljust(12),descrip.ljust(30),
rating.rjust(4),)
defaddtodb(name,descrip,rating):
try:
rating=int(rating)
exceptValueError,exp:
print'Error:ratingmustbeinteger.'
return
cursor.execute("select*fromplantsdbwherep_name
='%s'"%name)
iflen(rows)>0:
ql="updateplantsdbsetp_descrip='%s',
p_rating='%s'wherep_name='%s'"%(
descrip,rating,name,)
print'ql:',ql
cursor.execute(ql)
connection.commit()
print'Updated'
else:
cursor.execute("insertintoplantsdbvalues
('%s','%s','%s')"%(
name,descrip,rating))
Page203
connection.commit()
print'Added'
showdb1(cursor)
connection.close()
defdeletefromdb(name):
cursor.execute("select*fromplantsdbwherep_name
='%s'"%name)
iflen(rows)>0:
cursor.execute("deletefromplantsdbwhere
p_name='%s'"%name)
connection.commit()
print'Plant(%s)deleted.'%name
else:
print'Plant(%s)doesnotexist.'%name
showdb1(cursor)
connection.close()
defopendb():
connection=sqlite3.connect("sqlite3plantsdb")
cursor=connection.cursor()
returnconnection,cursor
defhr():
print''*60
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)<1:
usage()
cmd=args[0]
ifcmd=='create':
iflen(args)!=1:
usage()
createdb()
elifcmd=='show':
iflen(args)!=1:
usage()
showdb()
elifcmd=='add':
iflen(args)<4:
usage()
name=args[1]
descrip=args[2]
rating=args[3]
Page204
addtodb(name,descrip,rating)
elifcmd=='delete':
iflen(args)<2:
usage()
name=args[1]
deletefromdb(name)
else:
usage()
main()
3.9.3CSVcommaseparatedvaluefiles
ThereissupportforparsingandgeneratingCSVfilesinthePythonstandardlibrary.See:
csvCSVFileReadingandWritinghttp://docs.python.org/library/csv.html#module
csv.
Exercises:
1. ReadaCSVfileandprintthefieldsincolumns.Hereisasamplefiletouseas
input:
#namedescriptionrating
Lemon,Brightyellowandtart,5
Eggplant,Purpleandshiny,6
Tangerine,Succulent,8
Solutions:
1. UsetheCSVmoduleinthePythonstandardlibrarytoreadaCSVfile:
"""
ReadaCSVfileandprintthecontentsincolumns.
"""
importcsv
infile=open(infilename)
reader=csv.reader(infile)
print'===============
======'
print'NameDescription
Rating'
print'===============
======'
forfieldsinreader:
iflen(fields)==3:
line='%s%s%s'%(fields[0].ljust(20),
fields[1].ljust(40),
fields[2].ljust(4))
printline
infile.close()
defmain():
infilename='csv_report.csv'
Page205
test(infilename)
main()
And,whenrun,hereiswhatitdisplays:
===============
======
NameDescription
Rating
===============
======
LemonBrightyellowandtart
5
EggplantPurpleandshiny
6
TangerineSucculent
8
3.9.4YAMLandPyYAML
YAMLisastructuredtextdatarepresentationformat.Itusesindentationtoindicate
nesting.HereisadescriptionfromtheYAMLWebsite:
"YAML:YAMLAin'tMarkupLanguage
"WhatItIs:YAMLisahumanfriendlydataserializationstandardfor
allprogramminglanguages."
YoucanlearnmoreaboutYAMLandPyYAMLhere:
TheOfficialYAMLWebSitehttp://yaml.org/
PyYAML.orgthehomeofvariousYAMLimplementationsforPython
http://pyyaml.org/
TheYAML1.2specificationhttp://yaml.org/spec/1.2/
Exercises:
1. ReadthefollowingsampleYAMLdocument.Printouttheinformationinit:
american:
BostonRedSox
DetroitTigers
NewYorkYankees
national:
NewYorkMets
ChicagoCubs
AtlantaBraves
2. LoadtheYAMLdatausedinthepreviousexercise,thenmakeamodification(for
example,add"SanFranciscoGiants"totheNationalLeague),thendumpthe
modifieddatatoanewfile.
Solutions:
1. PrintingoutinformationfromYAMLisas"simple"asprintingoutaPythondata
structure.Inthissolution,weusetheprettyprinterfromthePythonstandard
Page206
library:
importyaml
importpprint
deftest():
infile=open('test1.yaml')
data=yaml.load(infile)
infile.close()
pprint.pprint(data)
test()
Wecould,alternatively,readinandthen"load"fromastring:
importyaml
importpprint
deftest():
infile=open('test1.yaml')
data_str=infile.read()
infile.close()
data=yaml.load(data_str)
pprint.pprint(data)
test()
2. TheYAMLdump()functionenablesustodumpdatatoafile:
importyaml
importpprint
deftest():
infile=open('test1.yaml','r')
data=yaml.load(infile)
infile.close()
data['national'].append('SanFranciscoGiants')
outfile=open('test1_new.yaml','w')
yaml.dump(data,outfile)
outfile.close()
test()
Notes:
IfwewanttoproducethestandardYAML"block"styleratherthanthe"flow"
format,thenwecoulduse:
yaml.dump(data,outfile,default_flow_style=False)
3.9.5Json
HereisaquotefromWikipediaentryforJson:
"JSON(pronounced'Jason'),shortforJavaScriptObjectNotation,isa
lightweightcomputerdatainterchangeformat.Itisatextbased,
humanreadableformatforrepresentingsimpledatastructuresand
associativearrays(calledobjects)."
TheJsontextrepresentationlooksverysimilartoPythonliteralrepresentationofPython
Page207
builtindatatypes(forexample,lists,dictionaries,numbers,andstrings).
LearnmoreaboutJsonandPythonsupportforJsonhere:
IntroducingJSONhttp://json.org/
JsonatWikipediahttp://en.wikipedia.org/wiki/Json
pythonjsonhttp://pypi.python.org/pypi/pythonjson
simplejsonhttp://pypi.python.org/pypi/simplejson
Excercises:
1. WriteaPythonscript,usingyourfavoritePythonJsonimplementation(for
examplepythonjsonorsimplejson),thatdumpsthefollowingdata
structuretoafile:
Data={
'rockandroll':
['Elis','TheBeatles','TheRollingStones',],
'country':
['WillieNelson','HankWilliams',]
}
2. WriteaPythonscriptthatreadsJsondatafromafileandloadsitintoPythondata
structures.
Solutions:
1. ThissolutionusessimplejsontostoreaPythondatastructureencodedasJson
inafile:
importsimplejsonasjson
Data={
'rockandroll':
['Elis','TheBeatles','TheRollingStones',],
'country':
['WillieNelson','HankWilliams',]
}
deftest():
fout=open('tmpdata.json','w')
content=json.dumps(Data)
fout.write(content)
fout.write('\n')
fout.close()
test()
2. Wecanreadthefileintoastring,thendecodeitfromJson:
importsimplejsonasjson
deftest():
fin=open('tmpdata.json','r')
content=fin.read()
fin.close()
data=json.loads(content)
printdata
Page208
test()
Notethatyoumaywantsomecontroloverindentation,characterencoding,etc.For
simplejson,youcanlearnaboutthathere:simplejsonJSONencoderanddecoder
http://simplejson.googlecode.com/svn/tags/simplejson2.0.1/docs/index.html.
Page209
4Part4GeneratingPythonBindingsforXML
ThissectiondiscussesaspecificPythontool,specificallyaPythoncodegeneratorthat
generatesPythonbindingsforXMLfiles.
Thus,thissectionwillhelpyouinthefollowingways:
1. Itwillhelpyoulearntouseaspecifictool,namelygenerateDS.py,that
generatesPythoncodetobeusedtoprocessXMLinstancedocumentsofa
particulardocumenttype.
2. Itwillhelpyougainmoreexperiencewithreading,modifyingandusingPython
code.
4.1Introduction
Additionalinformation:
Ifyouplantoworkthroughthistutorial,youmayfindithelpfultolookatthe
samplecodethataccompaniesthistutorial.Youcanfinditinthedistribution
under:
tutorial/
tutorial/Code/
YoucanfindadditionalinformationaboutgenerateDS.pyhere:
http://www.rexx.com/~dkuhlman/generateDS.html
Thatdocumentationisalsoincludedinthedistribution.
generateDS.pygeneratesPythondatastructures(forexample,classdefinitions)from
anXMLschemadocument.ThesedatastructuresrepresenttheelementsinanXML
documentdescribedbytheXMLschema.generateDS.pyalsogeneratesparsersthat
loadanXMLdocumentintothosedatastructures.Inaddition,aseparatefilecontaining
subclasses(stubs)isoptionallygenerated.Theusercanaddmethodstothesubclassesin
ordertoprocessthecontentsofanXMLdocument.
ThegeneratedPythoncodecontains:
AclassdefinitionforeachelementdefinedintheXMLschemadocument.
Amainanddriverfunctionthatcanbeusedtotestthegeneratedcode.
AparserthatwillreadanXMLdocumentwhichsatisfiestheXMLschemafrom
whichtheparserwasgenerated.Theparsercreatesandpopulatesatreestructure
ofinstancesofthegeneratedPythonclasses.
MethodsineachclasstoexporttheinstancebackouttoXML(methodexport)
andtoexporttheinstancetoaliteralrepresentingthePythondatastructure
(methodexportLiteral).
Eachgeneratedclasscontainsthefollowing:
Page210
Aconstructormethod(__init__),withmembervariableinitializers.
Methodswithnamesget_xyzandset_xyzforeachmembervariable"xyz"
or,ifthemembervariableisdefinedwithmaxOccurs="unbounded",
methodswithnamesget_xyz,set_xyz,add_xyz,andinsert_xyz.
(Note:Ifyouusetheuseoldgettersetter,thenyouwillget
methodswithnameslikegetXyzandsetXyz.)
Abuildmethodthatcanbeusedtopopulateaninstanceoftheclassfroma
nodeinanElementTreeorLxmltree.
Anexportmethodthatwillwritetheinstance(andanynestedsubinstances)to
afileobjectasXMLtext.
AnexportLiteralmethodthatwillwritetheinstance(andanynestedsub
instances)toafileobjectasPythonliterals(text).
Thegeneratedsubclassfilecontainsone(sub)classdefinitionforeachdata
representationclass.Ifthesubclassfileisused,thentheparsercreatesinstancesofthe
subclasses(insteadofcreatinginstancesofthesuperclasses).Thisenablestheuserto
extendthesubclasseswith"treewalk"methods,forexample,thatprocessthecontentsof
theXMLfile.Theusercanalsogenerateandextendmultiplesubclassfileswhichusea
single,commonsuperclassfile,thusimplementinganumberofdifferentprocessesonthe
sameXMLdocumenttype.
ThisdocumentintroducestheusertogenerateDS.pyandwalkstheuserthrough
severalexamplesthatshowhowtogeneratePythoncodeandhowtousethatgenerated
code.
4.2Generatingthecode
Note:Thesamplefilesusedbelowareunderthetutorial/Code/directory.
Usethefollowingtogethelp:
$generateDS.pyhelp
I'llassumethatgenerateDS.pyisinadirectoryonyourpath.Ifnot,youshoulddo
whateverisnecessarytomakeitaccessibleandexecutable.
HereisasimpleXMLschemadocument:
And,hereishowyoumightgenerateclassesandsubclassesthatprovidedatabindings(a
PythonAPI)forthedefinitionsinthatschema:
$generateDS.pyopeople_api.pyspeople_sub.pypeople.xsd
And,ifyouwanttoautomaticallyoverwritethegeneratedPythonfiles,usethef
commandlineflagtoforceoverwritewithoutasking:
$generateDS.pyfopeople_api.pyspeople_sub.pypeople.xsd
And,tohardwirethesubclassfilesothatitimportstheAPImodule,usethesuper
Page211
commandlinefile.Example:
$generateDS.pyopeople_api.pypeople.xsd
$generateDS.pyspeople_appl1.pysuper=people_apipeople.xsd
Or,dobothatthesametimewiththefollowing:
$generateDS.pyopeople_api.pyspeople_appl1.py
super=people_apipeople.xsd
And,foryoursecondapplication:
$generateDS.pyspeople_appl2.pysuper=people_apipeople.xsd
Ifyoutakealookinsidethesetwo"application"files,youwillseeandimportstatement
likethefollowing:
import???assupermod
Ifyouhadnotusedthesupercommandlineoptionwhengeneratingthe
"application"files,thenyoucouldmodifythatstatementyourself.Thesuper
commandlineoptiondoesthisforyou.
YoucanalsousetheThegraphicalfrontendtoconfigureoptionsandsavethemina
sessionfile,thenusethatsessionfilewithgenerateDS.pytospecifyyourcommand
lineoptions.Forexample:
$generateDS.pysession=test01.session
Youcantestthegeneratedcodebyrunningit.Trysomethinglikethefollowing:
$pythonpeople_api.pypeople.xml
or:
$pythonpeople_appl1.pypeople.xml
Whydoesthiswork?WhycanwerunthegeneratedcodeasaPythonscript?Ifyou
lookatthegeneratedcode,downneartheendofthefileyou'llfindamain()function
thatcallsafunctionnamedparse().Theparsefunctiondoesthefollowing:
1. ParsesyourXMLinstancedocument.
2. UsesyourgeneratedAPItobuildatreeofinstancesofthegeneratedclasses.
3. Usestheexport()methodsinthattreeofinstancestoprintout(export)XML
thatrepresentsyourgeneratedtreeofinstances.
Exceptforsomeindentation(ignorablewhitespace),thisexportedXMLshouldbethe
sameastheoriginalXMLdocument.So,thatgivesyouareasonablythoroughtestof
yourgeneratedcode.
And,thecodeinthatparse()functiongivesyouahintofhowyoumightbuildyour
ownapplicationspecificcodethatusesthegeneratedAPI(thosegeneratedPython
classes).
Page212
4.3UsingthegeneratedcodetoparseandexportanXMLdocument
Nowthatyouhavegeneratedcodeforyourdatamodel,youcantestitbyrunningitasan
application.SupposethatyouhaveanXMLinstancedocumentpeople1.xmlthat
satisfiesyourschema.Thenyoucanparsethatinstancedocumentandexportit(printit
out)withsomethinglikethefollowing:
$pythonpeople_api.pypeople1.xml
And,ifyouhaveusedthesupercommandlineoption,asIhaveabove,toconnect
yoursubclassfilewiththesuperclass(API)file,thenyoucouldusethefollowingtodo
thesamething:
$pythonpeople_appl1.pypeople1.xml
4.4Somecommandlineoptionsyoumightwanttoknow
Youmaywanttomerelyskimthissectionfornow,thenlaterreferbacktoitwhensome
oftheseoptionsareareusedlaterinthistutorial.Also,rememberthatyoucanget
informationaboutmorecommandlineoptionsusedbygenerateDS.pybytyping:
$pythongenerateDS.pyhelp
andbyreadingthedocumenthttp://www.rexx.com/~dkuhlman/generateDS.html
o
Generatethesuperclassmodule.Thisisthemodulethatcontainstheimplementation
ofeachclassforeachelementtype.So,youcanthinkofthisastheimplementationof
the"databindings"ortheAPIforXMLdocumentsofthetypedefinedbyyourXML
schema.
s
Generatethesubclassmodule.Youmightormightnotneedthese.Ifyouintendto
writesomeapplicationspecificcode,youmightwanttoconsiderstartingwiththese
skeletonclassesandaddyourapplicationcodethere.
super
Thisoptioninsertsthenameofthesuperclassmoduleintoanimportstatementin
thesubclassfile(generatedwith"s").Ifyouknowthenameofthesuperclassfilein
advance,youcanusethisoptiontoenablethesubclassfiletoimportthesuperclass
moduleautomatically.Ifyoudonotusethisoption,youwillneedtoeditthesubclass
modulewithyourtexteditorandmodifytheimportstatementnearthetop.
rootelement="elementname"
UsethisoptiontotellgenerateDS.pywhichoftheelementsdefinedinyourXM
schemaisthe"root"element.Therootelementistheoutermost(toplevel)element
inXMLinstancedocumentsdefinedbythisschema.Ineffect,thistellsyour
Page213
generatedmoduleswhichelementtouseastherootelementwhenparsingand
exportingdocuments.
generateDS.pyattemptstoguesstherootelement,usuallythefirstelement
definedinyourXMLschema.Usethisoptionwhenthatdefaultisnotwhatyouwant.
memberspecs=list|dict
Supposeyouwanttowritesomecodethatcanbegenericallyappliedtoelementsof
differentkinds(elementtypesimplementedbyseveraldifferentgeneratedclasses.If
so,itmightbehelpfultohavealistordictionaryspecifyinginformationabouteach
memberdataitemineachclass.Thisoptiondoesthatbygeneratingalistora
dictionary(withthememberdataitemnameaskey)ineachgeneratedclass.Takea
lookatthegeneratedcodetolearnaboutit.Inparticular,lookatthegeneratedlistor
dictionaryinaclassforanyelementtypeandalsoatthedefinitionoftheclass
_MemberSpecgeneratednearthetopoftheAPImodule.
version
AskgenerateDS.pytotellyouwhatversionitis.Thisishelpfulwhenyouwant
toaskaboutaproblem,forexampleatthegeneratedsusersemaillist
(https://lists.sourceforge.net/lists/listinfo/generatedsusers),andwanttospecifywhich
versionyouareusing.
4.5Thegraphicalfrontend
ThereisalsoapointandclickwaytorungenerateDS.Itenablesyoutospecifythe
optionsneededbygenerateDS.pythroughagraphicalinterface,thentorun
generateDS.pywiththoseoptions.Italso
Youcanrunit,ifyouhaveinstalledgenerateDS,bytypingthefollowingata
commandline:
$generateds_gui.py
Afterconfiguringoptions,youcansavethoseoptionsina"session"file,whichcanbe
loadedlater.LookundertheFilemenuforsaveandloadcommandsandalsoconsider
usingthe"session"commandlineoption.
AlsonotethatgenerateDS.pyitselfsupportsa"session"commandlineoptionthat
enablesyoutorungenerateDS.pywiththeoptionsthatyouspecifiedandsavedwith
thegraphicalfrontend.
4.6Addingapplicationspecificbehavior
generateDS.pygeneratesPythoncodewhich,withnomodification,willparseand
thenexportanXMLdocumentdefinedbyyourschema.However,youarelikelytowant
togobeyondthat.Inmanysituationsyouwillwanttoconstructacustomapplicationthat
processesyourXMLdocumentsusingthegeneratedcode.
Page214
4.6.1Implementingcustomsubclasses
Onestrategyistogenerateasubclassfileandtoaddyourapplicationspecificcodeto
that.Generatethesubclassfilewiththe"s"commandlineflag:
$generateDS.pysmyapp.pypeople.xsd
Nowaddsomeapplicationspecificcodetomyapp.py,forexample,ifyouareusingthe
included"people"samplefiles:
classpeopleTypeSub(supermod.people):
def__init__(self,comments=None,person=None,programmer=None,
python_programmer=None,java_programmer=None):
supermod.people.__init__(self,comments,person,programmer,
python_programmer,
java_programmer)
deffancyexport(self,outfile):
outfile.write('Startingfancyexport')
forpersoninself.get_person():
person.fancyexport(outfile)
supermod.people.subclass=peopleTypeSub
#endclasspeopleTypeSub
classpersonTypeSub(supermod.person):
def__init__(self,vegetable=None,fruit=None,ratio=None,
id=None,value=None,
name=None,interest=None,category=None,agent=None,
promoter=None,
description=None):
supermod.person.__init__(self,vegetable,fruit,ratio,id,
value,
name,interest,category,agent,promoter,description)
deffancyexport(self,outfile):
outfile.write('Fancypersonexportname:%s'%
self.get_name(),)
supermod.person.subclass=personTypeSub
#endclasspersonTypeSub
4.6.2Usingthegenerated"API"fromyourapplication
Inthisapproachyoumightdothingslikethefollowing:
importyourgeneratedclasses.
Createinstancesofthoseclasses.
Linkthoseinstances,forexampleput"children"insideofaparent,oraddoneor
moreinstancestoaparentthatcancontainalistofobjects(think"maxOccurs"
greaterthan1inyourschema)
GettoknowthegeneratedexportAPIbyinspectingthegeneratedcodeinthesuperclass
file.That'sthefilegeneratedwiththe"o"commandlineflag.
Whattolookfor:
Page215
Lookattheargumentstotheconstructor(__init__)tolearnhowtoinitialize
aninstance.
Lookatthe"getters"and"setters"(methodsnamegetxxxandsetxxx,tolearn
howtomodifymembervariables.
Lookforamethodnamedaddxxxformembersthatarelists.Thesecorrespond
tomembersdefinedwithmaxOccurs="n",wheren>1.
Lookatthebuildmethods:build,buildChildren,and
buildAttributes.Thesewillgiveyouinformationabouthowtoconstruct
eachofthemembersofagivenelement/class.
Now,youcanimportyourgeneratedAPImodule,anduseittoconstructandmanipulate
objects.Hereisanexampleusingcodegeneratedwiththe"people"schema:
importsys
importpeople_apiasapi
deftest(names):
people=api.peopleType()
forcount,nameinenumerate(names):
id='%d'%(count+1,)
person=api.personType(name=name,id=id)
people.add_person(person)
people.export(sys.stdout,0)
test(['albert','betsy','charlie'])
Runthisandyoumightseesomethinglikethefollowing:
$pythontmp.py
<people>
<personid="1">
<name>albert</name>
</person>
<personid="2">
<name>betsy</name>
</person>
<personid="3">
<name>charlie</name>
</person>
</people>
4.6.3Acombinedapproach
Note:Youcanfindexamplesofthecodeinthissectioninthesefiles:
tutorial/Code/upcase_names.py
tutorial/Code/upcase_names_appl.py
Herearetherelevant,modifiedsubclasses(upcase_names_appl.py):
importpeople_apiassupermod
Page216
classpeopleTypeSub(supermod.peopleType):
def__init__(self,comments=None,person=None,
specialperson=None,programmer=None,python_programmer=None,
java_programmer=None):
super(peopleTypeSub,self).__init__(comments,person,
specialperson,programmer,python_programmer,java_programmer,)
defupcase_names(self):
forpersoninself.get_person():
person.upcase_names()
supermod.peopleType.subclass=peopleTypeSub
#endclasspeopleTypeSub
classpersonTypeSub(supermod.personType):
def__init__(self,vegetable=None,fruit=None,ratio=None,
id=None,value=None,name=None,interest=None,category=None,
agent=None,promoter=None,description=None,range_=None,
extensiontype_=None):
super(personTypeSub,self).__init__(vegetable,fruit,ratio,
id,value,name,interest,category,agent,promoter,description,
range_,extensiontype_,)
defupcase_names(self):
self.set_name(self.get_name().upper())
supermod.personType.subclass=personTypeSub
#endclasspersonTypeSub
Notes:
Theseclassesweregeneratedwiththe"s"commandlineoption.Theyare
subclassesofclassesinthemodulepeople_api,whichwasgeneratedwiththe
"o"commandlineoption.
Theonlymodificationtotheskeletonsubclassesistheadditionofthetwo
methodsnamedupcase_names().
InthesubclasspeopleTypeSub,themethodupcase_names()merelywalk
overitsimmediatechildren.
InthesubclasspersonTypeSub,themethodupcase_names()justconverts
thevalueofits"name"membertouppercase.
Hereistheapplicationitself(upcase_names.py):
importsys
importupcase_names_applasappl
defcreate_people(names):
people=appl.peopleTypeSub()
forcount,nameinenumerate(names):
id='%d'%(count+1,)
person=appl.personTypeSub(name=name,id=id)
people.add_person(person)
returnpeople
defmain():
names=['albert','betsy','charlie']
people=create_people(names)
print'Before:'
Page217
people.upcase_names()
print''*50
print'After:'
main()
Notes:
Thecreate_people()functioncreatesapeopleTypeSubinstancewith
severalpersonTypeSubinstancesinsideit.
And,whenyourunthisminiapplication,hereiswhatyoumightsee:
$pythonupcase_names.py
Before:
<people>
<personid="1">
<name>albert</name>
</person>
<personid="2">
<name>betsy</name>
</person>
<personid="3">
<name>charlie</name>
</person>
</people>
After:
<people>
<personid="1">
<name>ALBERT</name>
</person>
<personid="2">
<name>BETSY</name>
</person>
<personid="3">
<name>CHARLIE</name>
</person>
</people>
4.7Specialsituationsanduses
4.7.1Generic,typeindependentprocessing
Therearetimeswhenyouwouldliketoimplementafunctionormethodthatcanperform
operationsonavarietyofmembersandthatneedstypeinformationabouteachmember.
Youcangethelpwiththisbygeneratingyourcodewiththe"memberspecs"command
lineoption.Whenyouusethisoption,generateDS.pyaddalistoradictionary
containinganitemforeachmember.Ifyouwantalist,thenuse"memberspecs=list",
andifyouwantadictionary,withmembernamesaskeys,thenuse"member
Page218
specs=dict".
HereisanexampleInthisexample,wewalkthedocument/instancetreeandconvert
allstringsimpletypestouppercase.
Hereisaschema(Code/member_specs.xsd):
<xs:schemaxmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:elementname="contactlist"type="contactlistType"/>
<xs:complexTypename="contactlistType">
<xs:sequence>
<xs:elementname="description"type="xs:string"/>
<xs:elementname="contact"type="contactType"
maxOccurs="unbounded"/>
</xs:sequence>
<xs:attributename="locator"type="xs:string"/>
</xs:complexType>
<xs:complexTypename="contactType">
<xs:sequence>
<xs:elementname="firstname"type="xs:string"/>
<xs:elementname="lastname"type="xs:string"/>
<xs:elementname="interest"type="xs:string"
<xs:elementname="category"type="xs:integer"/>
</xs:sequence>
<xs:attributename="id"type="xs:integer"/>
<xs:attributename="priority"type="xs:float"/>
<xs:attributename="colorcode"type="xs:string"/>
</xs:complexType>
</xs:schema>
4.7.1.1Step1generatethebindings
Wegeneratecodewiththefollowingcommandline:
$generateDS.pyf\
omember_specs_api.py\
smember_specs_upper.py\
super=member_specs_api\
memberspecs=list\
member_specs.xsd
Notes:
Wegeneratethememberspecificationsasalistwiththecommandlineoption
memberspecs=list.
Wegeneratean"application"modulewiththescommandlineoption.We'llput
ourapplicationspecificcodeinmember_specs_upper.py.
Page219
4.7.1.2Step2addapplicationspecificcode
And,hereisthesubclassfile(member_specs_upper.py,generatedwiththe"s"
commandlineoption),towhichwehaveaddedabitofcodethatconvertsanystringtype
memberstouppercase.Youcanthinkofthismoduleasaspecial"application"ofthe
generatedclasses.
#
#member_specs_upper.py
#
#
#GeneratedTueNov915:54:472010bygenerateDS.pyversion2.2a.
#
importsys
importmember_specs_apiassupermod
etree_=None
Verbose_import_=False
(XMLParser_import_none,XMLParser_import_lxml,
XMLParser_import_elementtree
)=range(3)
XMLParser_import_library=None
try:
#lxml
fromlxmlimportetreeasetree_
XMLParser_import_library=XMLParser_import_lxml
ifVerbose_import_:
print("runningwithlxml.etree")
exceptImportError:
try:
#cElementTreefromPython2.5+
importxml.etree.cElementTreeasetree_
XMLParser_import_library=XMLParser_import_elementtree
ifVerbose_import_:
print("runningwithcElementTreeonPython2.5+")
exceptImportError:
try:
#ElementTreefromPython2.5+
importxml.etree.ElementTreeasetree_
XMLParser_import_library=XMLParser_import_elementtree
ifVerbose_import_:
print("runningwithElementTreeonPython2.5+")
exceptImportError:
try:
#normalcElementTreeinstall
importcElementTreeasetree_
XMLParser_import_library=
ifVerbose_import_:
print("runningwithcElementTree")
Page220
exceptImportError:
try:
#normalElementTreeinstall
importelementtree.ElementTreeasetree_
XMLParser_import_library=
ifVerbose_import_:
print("runningwithElementTree")
exceptImportError:
raiseImportError("FailedtoimportElementTree
fromanyknownplace")
defparsexml_(*args,**kwargs):
if(XMLParser_import_library==XMLParser_import_lxmland
'parser'notinkwargs):
#UsethelxmlElementTreecompatibleparsersothat,e.g.,
#weignorecomments.
kwargs['parser']=etree_.ETCompatXMLParser()
doc=etree_.parse(*args,**kwargs)
returndoc
#
#Globals
#
ExternalEncoding='ascii'
#
#Utilityfuntionsneededineachgeneratedclass.
#
defupper_elements(obj):
foriteminobj.member_data_items_:
ifitem.get_data_type()=='xs:string':
name=remap(item.get_name())
val1=getattr(obj,name)
ifisinstance(val1,list):
foridx,val2inenumerate(val1):
val1[idx]=val2.upper()
else:
setattr(obj,name,val1.upper())
defremap(name):
newname=name.replace('','_')
returnnewname
#
#Datarepresentationclasses
#
classcontactlistTypeSub(supermod.contactlistType):
def__init__(self,locator=None,description=None,contact=None):
super(contactlistTypeSub,self).__init__(locator,
description,contact,)
defupper(self):
Page221
upper_elements(self)
forchildinself.get_contact():
child.upper()
supermod.contactlistType.subclass=contactlistTypeSub
#endclasscontactlistTypeSub
classcontactTypeSub(supermod.contactType):
def__init__(self,priority=None,color_code=None,id=None,
first_name=None,last_name=None,interest=None,category=None):
super(contactTypeSub,self).__init__(priority,color_code,
id,first_name,last_name,interest,category,)
defupper(self):
upper_elements(self)
supermod.contactType.subclass=contactTypeSub
#endclasscontactTypeSub
defget_root_tag(node):
tag=supermod.Tag_pattern_.match(node.tag).groups()[1]
rootClass=None
ifhasattr(supermod,tag):
rootClass=getattr(supermod,tag)
returntag,rootClass
defparse(inFilename):
doc=parsexml_(inFilename)
rootNode=doc.getroot()
rootTag,rootClass=get_root_tag(rootNode)
ifrootClassisNone:
rootTag='contactlist'
rootClass=supermod.contactlistType
rootObj=rootClass.factory()
rootObj.build(rootNode)
#EnablePythontocollectthespaceusedbytheDOM.
doc=None
sys.stdout.write('<?xmlversion="1.0"?>\n')
rootObj.export(sys.stdout,0,name_=rootTag,
namespacedef_='')
doc=None
returnrootObj
defparseString(inString):
fromStringIOimportStringIO
doc=parsexml_(StringIO(inString))
ifrootClassisNone:
doc=None
Page222
rootObj.export(sys.stdout,0,name_=rootTag,
namespacedef_='')
returnrootObj
defparseLiteral(inFilename):
doc=parsexml_(inFilename)
ifrootClassisNone:
doc=None
sys.stdout.write('#frommember_specs_apiimport*\n\n')
sys.stdout.write('importmember_specs_apiasmodel_\n\n')
sys.stdout.write('rootObj=model_.contact_list(\n')
rootObj.exportLiteral(sys.stdout,0,name_="contact_list")
sys.stdout.write(')\n')
returnrootObj
USAGE_TEXT="""
Usage:python???.py<infilename>
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)!=1:
usage()
infilename=args[0]
root=parse(infilename)
main()
Notes:
Weaddthefunctionsupper_elementsandremapthatweuseineach
generatedclass.
Noticehowthefunctionupper_elementscallsthefunctionremaponlyon
thosememberswhosetypeisxs:string.
Ineachgenerated(sub)class,weaddthemethodsthatwalktheDOMtreeand
applythemethod(upper)thattransformseachxs:stringvalue.
Page223
4.7.1.3Step3writeatest/driverharness
Hereisatestdriver(member_specs_test.py)forour(mini)application:
#
#member_specs_test.py
#
importsys
importmember_specs_apiassupermod
importmember_specs_upper
defprocess(inFilename):
doc=supermod.parsexml_(inFilename)
rootClass=member_specs_upper.contactlistTypeSub
doc=None
rootObj.export(sys.stdout,0,name_="contactlist",
namespacedef_='')
rootObj.upper()
sys.stdout.write(''*60)
sys.stdout.write('\n')
rootObj.export(sys.stdout,0,name_="contactlist",
namespacedef_='')
returnrootObj
USAGE_MSG="""\
Synopsis:
Sampleapplicationusingclassesandsubclassesgeneratedby
generateDS.py
Usage:
pythonmember_specs_test.pyinfilename
"""
defusage():
printUSAGE_MSG
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)!=1:
usage()
infilename=args[0]
process(infilename)
main()
Page224
Notes:
Wecopythefunctionparse()fromourgeneratedcodetoserveasamodelfor
ourfunctionprocess().
AfterparsinganddisplayingtheXMLinstancedocument,wecallmethod
upper()inthegeneratedclasscontactlistTypeSubinordertowalkthe
DOMtreeandtransformeachxs:stringtouppercase.
4.7.1.4Step4runthetestapplication
Wecanusethefollowingcommandlinetorunourapplication:
$pythonmember_specs_test.pymember_specs_data.xml
Whenwerunourapplication,hereistheoutput:
$pythonmember_specs_test.pymember_specs_data.xml
<contactlistlocator="http://www.rexx.com/~dkuhlman">
<description>Mylistofcontacts</description>
<contactpriority="0.050000"colorcode="red"id="1">
<firstname>arlene</firstname>
<lastname>Allen</lastname>
<interest>traveling</interest>
</contact>
</contactlist>
<contactlistlocator="HTTP://WWW.REXX.COM/~DKUHLMAN">
<description>MYLISTOFCONTACTS</description>
<contactpriority="0.050000"colorcode="RED"id="1">
<firstname>ARLENE</firstname>
<lastname>ALLEN</lastname>
<interest>TRAVELING</interest>
</contact>
</contactlist>
Notes:
Theoutputaboveshowsbothbeforeandafterversionofexportingtheparsed
XMLinstancedocument.
4.8Somehints
Thefollowinghintsareofferedforconvenience.Youcandiscoverthemforyourself
rathereasilybyinspectingthegeneratedcode.
4.8.1ChildrendefinedwithmaxOccursgreaterthan1
IfachildelementisdefinedintheXMLschemawithmaxOccurs="unbounded"or
avalueofmaxOccursgreaterthan1,thenaccesstothechildisthroughalist.
Page225
4.8.2Childrendefinedwithsimplenumerictypes
Ifachildelementisdefinedasanumerictypesuchasxs:integer,xs:float,or
xs:doubleorasasimpletypethatis(ultimately)basedonanumerictype,thenthe
valueisstored(inthePythonobject)asaPythondatatype(int,float,etc).
4.8.3Thetypeofanelement'scharactercontent
But,whentheelementitselfisdefinedasmixed="true"ortheelementarestrictionof
andhasasimple(numeric)asabase,thenthevalueOf_instancevariableholdsthe
charactercontentanditisalwaysastring,thatisitisnotconverted.
4.8.4Constructorsandtheirdefaultvalues
Allparameterstotheconstructorsofgeneratedclasseshavedefaultparameters.
Therefore,youcancreatean"empty"instanceofanyelementbycallingtheconstructor
withnoparameters.
Forexample,supposewehavethefollowingXMLschema:
<xs:schemaxmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:elementname="plantlist"type="PlantList"/>
<xs:complexTypename="PlantType">
<xs:sequence>
<xs:elementname="catagory"type="xs:integer"/>
<xs:elementname="fertilizer"type="FertilizerType"
</xs:sequence>
<xs:attributename="identifier"type="xs:string"/>
</xs:complexType>
<xs:complexTypename="FertilizerType">
<xs:sequence>
<xs:elementname="name"type="xs:string"/>
</xs:sequence>
<xs:attributename="id"type="xs:integer"/>
</xs:complexType>
</xs:schema>
And,supposewegenerateamodulewiththefollowingcommandline:
$./generateDS.pyogarden_api.pygarden.xsd
Then,fortheelementnamedPlantTypeinthegeneratedmodulenamed
garden_api.py,youcancreateaninstanceasfollows:
Page226
>>>importgarden_api
>>>plant=garden_api.PlantType()
>>>importsys
>>>plant.export(sys.stdout,0)
<PlantType/>
Page227

A Python Book Beginning Python, Advanced Python, and

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

A Python Book Beginning Python, Advanced Python, and

Transféré par

Droits d'auteur :

Formats disponibles

APythonBook:BeginningPython,AdvancedPython,and

Vous aimerez peut-être aussi