Preface

- Edward Gibbon

My motivation for writing this book grew out of a perceivedneed for an . integratedexplanatorytext on the subject This perceptioncame from the Robots I havehadin finding a goodbookfor a coursein Autonomous difficulty I havebeenteachingfor the pastten yearsor so. Although an extensive body of literaturehas beenproducedon behaviorbasedrobots the lack of sucha , to text madeit hardto introducestudents this field without throwing theminto technicalliterature generallymakingit accessible , only to advanced fairly deep students Thoughthere are severalgood books of collectedoriginal . graduate . as , papers they provedto be only partially adequate an introduction and This book' s intended audienceincludes upperlevel undergraduates studentsstudying artificial intellegence(AI ) and robotics as well , graduate the . in as thoseinterested learningmore aboutroboticsin general It assumes in college level artificial intelligence The text . a ability to comprehend course robotics or to could be usedto supporta coursein AI -basedor autonomous . a generalAI course supplement . , , Acknowledgmentsof course area necessityBeing a religiousman, I' d first like to thank God andJesusChrist for enablingme to completethis long and \\10biblical passages haveinspired me as a roboticist can that arduoustask. 1 be found in Matthew3:9 andLuke 19:37- 40. in generated My family hasbeenmostgracious putting up with the hardships . My wife, to my amazement , proofreadthe entirework. Sheand by authorship and by my four childrenput up with th~ inevitableabsences strainscaused this . . time-sappingprocessI love themdearlyfor their support

Preface

Earlier tutorialson behaviorbased robotics presented , initially by myself (at the 1991IEEE InternationalConference Systems Man, and Cybernetics on , ) and later with Rod Grupen(at the 1993lntemationalConference Robotics on and Automation and the 1993 InternationalJoint Conferenceon Artificial , . ) Intelligence helpedcoalesce manyof the ideascontainedherein I thank Rod for working with me on these . A large number of studentsat Georgia Tech over many years have contributed in a wide rangeof invaluableways. I ' d expresslylike to thank Robin , Murphy, Doug MacKenzie Thcker Balch, Khaled Ali , Zhong Chen Russ , Clark, ElizabethNitz, David Hobbs WarrenGardner William Carter Gary , , , Boone Michael PearceJuanCarlosSantamariaDavid Cardoze Bill Wester , , , , , Keith Ward David Vaughn Mark Pearson and others who have madethis , , , book possible I ' d alsolike to thankBrandonRhodes pointing out the short . for story that leadsinto chapter10. Interactionswith manyof my professional in at colleagues residence Georgia Tech aswell asvisitors, havealsohelpedto generate the ideasfound in of , this text. AmongthemareProf. ChrisAtkeson Dr. Jonathan CameronDr. Tom , , Collins, Prof. Ashok Ooel, Prof. Jessica , Hodgins Dr. Daryl Lawton, Dr. John Pani Prof. T. M . Rao, andProf. Ashwin Ram. My thanksto y ' all. , This book would never have been possiblewithout the researchfunding . ' eachof supportthat camefrom a variety of sources I d like to acknowledge theseagencies the cognizantfunding agent the National Science and : Foundation Khosla the Office (HowardMoraff), DARPA (Eric Mettala andPradeep ), of Naval Research Teresa McMullen), andthe Westing houseSavannah River ( Center(Clyde Ward . ) Technology The folks at MIT Presshave been great to work with from the book' s . , inception in particularthe late Harry Stantonand Jerry Weinstein I am also indebtedto Prof. Michael Arbib for so graciouslycontributingthe forewordto this book. Specialthanksgo to Jim Hendierfor piloting a draft versionof the book for a courseat the University of Maryland. Finally, it is impossibleto list all the people within the greaterresearch , support and insight , community who have contributedwith encouragement into this endeavorTo all of you I am deeplyindebtedand can only hopethat . this text serves someway ascompensation thoseefforts. in for

Foreword Michael Arbib

Had he turnedfrom politics to robotics John FitzgeraldKennedymight well , have said "Ask not what robotics can do for you, ask what you can do for , robotics" Ronald Arkin has a more balancedview, however for his book . , makesus vividly awarethat the interplaybetweenroboticsanda host of other . richly andfruitfully in both directions disciplinesproceeds " " Theemphasis on the" brains of robotsratherthanon their " bodies - thus is " which movesthe detailsof robot sensors " BehaviorBased and thetitle robots , to actuators , firmly into the backgroundmakingthem secondary the main aim what behaviorswe shouldexpectrobotsto exhibit, of the book: to understand . can and which computationalmechanisms serveto achievethesebehaviors featuresof the book that contributegreatly to its However I shouldnote two , of liveliness a fine selectionof epigramsanda superbgallery of photographs : robotsfrom all aroundthe world. Thus evenwhile the book focuseson robot . to brains we havemanyopportunities admirehumanwit androbot bodies , RonaldArkin hasalwaysbeenfascinated parallelsbetweenanimals(including by herein his marshalingof . humans and robots This is fully expressed ) datafrom biology and psychologyto showthat much of the behaviorof animals from a in andthusof robotscanbe understood termsof patterns emerging . basicsetof reactivemodules However Arkin also showsus that manybehaviors , of canbe achieved only by detailedanalysisof representations the world, 's -term experienceWe are . andof the animal placewithin it , built up from long that thusofferedan insightful analysisof robot architectures placesthemalong . a continuumfrom reactiveto deliberative a powerful frameworkexploitedhereto createbehaviors Schema theory provides that are distributed in their control structures integrativeof action , can . and perception and open to learning Theseschemas in turn be implemented , conventionalcomputerprograms finite stateacceptors neural , , using

Foreword networks or genetic algorithms In building our appreciationof this framework . , , Arkin usesroboticsto illuminate basic issuesin computerscienceand artificial intelligence andto feednewinsightsbackinto our readingof biology , andpsychology . The book' s chapteron social behaviorpresentswhat is itself a relatively recentchapterof robotics- socioroboticsWhile in its infancy we can seein . , the studiesof robot teams inter-robot communication and social lerning the , , , beginningsnot only of a powerful new technology but also of a new science of experimental . sociology Finally, we are takento that meetingplacebetweensciencefiction, philos, ophy and technologythat attractedmany of us to wonderaboutrobotsin the first place The final chapter " Fringe Robotics BeyondBehavior (a nod to . : " , ' " the 1960s British review " Beyondthe Fringe ?), debates issuesof robot the ' , , emotion and imagination returns to Arkin s longstanding , , thought consciousness concernwith the possibleutility to robots of analogsof hormones andhomeostasis closeswith an all too brief glimpseof nanotechnology . , and In this way we aregiven a tour that impress with the depthof its analysis es of the schemas , underlying robot behavior while continually illustrating the betweenrobotics and biology, psychology sociology and , , deep reciprocity betweenrobotics and many other , philosophy and the important connections areas computerscienceThis is a subjectwhosefascination only increase of . can in the decades aheadas many researchers build on the framework so ably here presented .

Chapter Whence

1 Behavior ?

Chapte Object 1thecon .unders Tospec what robot are .deve intellig 2andof a review recent that ledb tome the beh histor robotic . the.system .learn 3apprec To of wide robo
1.1 TOWARD ROBOTS INTELLIGENT
the : Perhaps bestway to begin our study is with a question If we could create robots what shouldthey be like, and what shouldthey be able to , intelligent do? Answeringthe first part of this question " What shouldthey be like?" 's ) requiresa descriptionof both the robot physical structure(appearanceand its performance of the question " What , (behavior. However the second ) part should they be able to do?" - frames the answerfor the first part. Robots that need to move objects must be able to grasp them; robots that have to traverserugged outdoor terrain need locomotion systemscapableof moving in adverseconditions robots that must function at night need sensors ; of operatingunder thoseconditions A guiding principle in robotic . capable the , , design whetherstructuralor behavioral involvesunderstanding environment within which the robot operatesand the task(s) it is required to undertake . This ecologicalapproach in which the robot' s goals and surroundings , influenceits design will be a recurring theme throughoutthis , heavily book. But what is a robot? Accordingto the RoboticsIndustryAssociation(RIA ), " a robot is a reto , programmablemultifunctional , manipulator designed move material parts tools, or specializeddevicesthrough variable programmed , , " motionsfor the performance a varietyof tasks (JablonskiandPosey1985 . of )

Chapterl

(A) Figure1.1 robots U musician of Anthropomorphic . (A) WAS BaT, a keyboard capable reading music (B) WlnrI , a robotthat walkedin excess 65 kin duringoneexhibition . of . of of .) ( photographs courtesy AtsuoTakanishi Waseda University This definition is quite restrictive excluding mobile robots among other , , ' robotics as the intelligent , things. On the other exb eme anotherdefinition describes connectionof perceptionto action ( Brady 1985 . This seems ) overly inclusive but doesacknowledge necessary the relationshipbetweentheseessential of . ingredients robotic systems In anycase our working definition will be: An intelligentrobot is a machine , able to extractinformation from its environmentand useknowledgeaboutits world to move safely in a meaningfuland purposivemanner Hollywood has .

? WhenceBehavior

(B) ) Figure 1.1 (continued

creaturesfashionedin the image often depictedrobots as anthropomorphic . two arms a torso, and a head Indeedrobots have of man having two legs , , , : e structure Figure 1.1 illustrates actually been createdthat have a hu~ . two such robots Robots have often been modeledafter animals other than and are commercially . humans however Insectlike robots are now commonplace , otherslook more like horses spiders or octopi, as figure 1.2 , available , ; . shows , Robotsalso often look like vehiclescapableof operatingon the ground in of classes . the air, or underseasThe examplesshownin figure 1.3 represent aerial : vehicles UAV (unmanned referredto as unmanned robots generically vehicle UGV (unmannedground vehicle and UUV (unmannedundersea ), ), vehicle . )

Figure 1.2

? WhenceBehavior

(C) ) Figure1.2(continued of robot (Photograph . crablike . Animallike robots (A) Ariel, a hexapod courtesy IS and robot , Robotics , , MA. ) , Somerville ) ( B Quadruped thattrots paces boundsbuilt at . . (Q MarcRaibert . in theCMUleglaboratory 1984(Photograph Jack Bingham 1992 by of robot .) All rightsreserved(C) HEGI060 courtesy California Hexapod . (Photograph Co. , CA. Cybernetics , Thjunga ) Robotscanbe differentiatedin termsof their size the materialsfrom which , aremade the way they arejoined together the actuators , they use(motors , they andtransmissions the typesof sensing , their locomotion theypossess ), systems . , system andtheir onboardcomputersystemsBut a physicalstructureis clearly . not enough Robotsmustbe animate so they musthavean underlyingcontrol , to providethe ability to movein a coordinated . This book focuses way system of roboticsandthe designof control and on theperformance behavioral aspects that systems allow themto performthe way we would like. The physicaldesign : Many goodsources of robotsis not addressed alreadycoverthat material(e.g., , ) Craig 1989 McKerrow 1991 .

Chapter1

(A)

(B ) 1.3 Figure

? WhenceRehavinr

(C) ) Figure 1. 3 (continued Search vehicle: the Advanced Unmanned vehicles (A ) Unmanned . undersea Unmanned lanceandSecurity : aerialvehicle MultipurposeSurveil ) System(AUSS . ( B) Unmanned : Mission Platform (MSSMP . (C) UnmannedGround Vehicle Ground Surveillance ) of U.S. Navy.) Robot (GSR . ( Photographs ) courtesy How do we realize the goal of intelligent robotic behavior? What basic science and technology is needed to achieve this goal ? This book attempts to answer these questions by studying the basis and organization of behavior and the related roles of knowledge and perception , learning and adaptation, and teamwork .

1.2 PRECURSORS
Peoplethat are really weird can get into sensitivepositions and have a tremendous impacton history. - J. DanforthQuayle

ChapterI To inventyou needa goodimaginationanda pile of junk . - ThomasAlva Edison

The significanthistory associated with the origins of modembehaviorbased roboticsis importantin understanding appreciating currentstateof the and the art. We now review important historical developments threerelatedareas in : . , , cyberneticsartificial intelligence androbotics

1.2.1

Cybernetics

Norbert Wiener is generallycreditedwith leading in the late 1940s the development , , of cybernetics a marriageof control theory information science : , , and biology that seeksto explain the commonprinciplesof control and communication in both animalsand machines( Wiener1948. Ashby ( 1952 and ) ) Wiener furtheredthis view of an organismas a machineby using the mathematics for . developed feedbackcontrol systemsto expressnatural behavior This affirmed the notion of situatednessthat is, a strong two-way coupling , betweenan organismandits environment . In 1953 W. Grey Walter applied theseprinciples in the creationof a precursor , robotic design termedMachina Speculatrix which was subsequently ' transformedinto hardwareform as Grey Walters tortoise Someof the principles . that werecapturedin his designinclude: 1. Parsimony Simple is better Simple reflexes can serveas the basisfor behavior : . . "The variationsof behaviorpatterns exhibitedevenwith sucheconomy " of structurearecomplexandunpredictable ( Walter 1953 p. 126 . , ) 2. Exploration or speculation The systemnever remainsstill except when : undernonnal circumstances ) feeding(recharging. This constantmotion is adequate to keepit from being trapped " In its explorationof any ordinaryroom . it inevitablyencounters ; manyobstaclesbut apartfrom stairsandfur rugs, there arefew situationsfrom which it cannotextricateitself ' ( Walker 1953 p. 126 . , ) 3. Attraction (positive tropism : The systemis motivatedto move towards ) someenvironmental of , object. In thecase thetortoise this is a light of moderate . intensity 4. Aversion(negativetropism : The systemmovesawayfrom certainnegative ) stimuli, for example avoidingheavyobstacles slopes and . , 5. Discernment The systemhasthe ability to distinguishbetweenproductive : andunproductive behavior adaptingitself to the situationat hand . ,

WhenceBehavior ?

. .

Figure 1.4 Circuit of MachinaSpeculatrix ( FromTheliving Brain by W. Grey Walter Copyright . . 1953@ 1963and renewed 1981 1991by W. Grey Walter Reprintedby permission @ . , of W. W. Norton andCompanyInc.) ,

The tortoise itself, constructedas an analogdevice (figure 1.4), consisted of two sensorstwo actuators and two " nervecells" or vacuumtubes A directional . , , for detectinglight and a bumpcontactsensor photocell providedthe environmental feedback One motor steered single front driving . the requisite wheel The photocell alwayspointed in the direction of this wheel and thus . could scanthe environment The driving motor poweredthe wheel and provided . locomotion . The tortoiseexhibitedthe following behaviors : . Seeking light : The sensorrotated until a weak light sourcewas detected while thedrive motorcontinuouslymovedtherobot to explorethe environment at the sametime. . Head toward weak light : Once a weak light was detected the tortoise , movedin its direction. . Back away from bright light : An aversivebehaviorrepelledthe tortoise from bright light sourcesThis behaviorwasusedto particularadvantage . when the tortoisewasrecharging . . Thm and push: Used to avoid obstacles this behavioroverrodethe light , . response . Rechargebattery : When the onboardbattery power was low, the tortoise the a perceived stronglight sourceas weak. Because rechargingstationhad a

. \

. 1 . ' : . .

Chapter1

Figure 1. 5 ' , Grey Walter s tortoise recently restoredto working order by Owen Holland., ( photo . graphcourtesyof OwenHolland, The University of the Westof England)

. , strong light over it , the robot movedtoward it and docked After recharging the light sourcewas perceivedas strong and the robot was repelledfrom the , . rechargingstation The behaviorswere prioritized from lowest to highest order: seekinglight, moveto/from light, andavoid obstacleThe tortoisealwaysactedon the highest . over , priority behaviorapplicable for examplechoosingto avoid obstacles toward a light . Behaviorbasedroboticsstill usesthis basicprinciple, moving referredto as an arbitration coordinationmechanism section3.4.3), widely. ( Walter' s tortoiseexhibitedmoderately : complexbehavior moving safelyabout a room and rechargingitself as needed(figure 1.5). One recent architecture ' in (Agah andBekey 1997 described section9.8.3, employsWalter s ideason ), positiveand negativetropismsas a basisfor creatingadaptivebehaviorbased robotic systems . Valentino Braitenberg revived this tradition three decadesafter Walter , ) (Braitenberg1984 . Taking the vantagepoint of a psychologist he extended the principles of analogcircuit behaviorto a seriesof gedankenexperiments . usedinhibitory involving the designof a collection of vehicles Thesesystems

? WhenceBehavior

. to and excitatoryinfluences directly coupling the sensors the motors As before , , seemingly complexbehaviorresultedfrom relativelysimplesensorimotor createda wide rangeof vehicles including those . transformationsBraitenberg , to exhibit cowardice aggressionandevenlove (figure 1.6). As with , , imagined Walter' s tortoise thesesystemsare inflexible, custommachinesand are not , . , reprogrammableNonethelessthe variability of their overt behavioris com pelling. , , Eventually scientistscreatedBraitenbergcreaturesthat were true robots ' at . not merely thoughtexperimentsIn one sucheffort, scientists MIT s Media Lab ( Hogg Martin, and Resnick 1991 usedspeciallymodified LEGO bricks ) , ' creaturevehiclesusing Braitenbergs principles to build twelve autonomous , a timid shadowseekeran indecisiveshadowedgefinder, a paranoid , including shadowfearing robot, a doggedobstacleavoider an insecurewall follower, , havebeenassembled seekerEven morecomplexcreatures . and a driven light ,

\" '.\ ., -I,,'"\ 0I I-

"

(A )

(B)

Figure 1.6 Vehicles Braitenberg / ) (A ) Vehicle1 (Singlemotor singlesensor: Motion is alwaysforward in the directionof . the sensorstalk, with the speedcontrolled by the sensorEnvironmentalperturbations in terrain producechanges direction. , ) (slippage rough /two motors : The photophobeon the left is aversiveto ) (B) Vehicle 2 ( Two sensors " ) light (exhibiting fear" by fleeing since the motor closestto the light sourcemoves . fasterthan the one farther away This resultsin a net motion awayfrom the light . The and sensors motors to on the left is attracted light whenthe wires connecting photovore " " from the photophobe aremerelyreversed (exhibiting aggression by charginginto the attractor . )

Chapter1

-'.\ II-: .0I\" I -

(C)
Figure 1.6 (continued ) (C) Vehicle3: Samewiring as for Vehicle2 but now with inhibitory connectionsThe . vehiclesslow down in the presence a stronglight sourceandgo fast in the presence of of weaklight . In both casesthe vehicleapproach and stopsby ilie light source(with es , one facing the light and one with the light sourceto the rear . The vehicle on the left ) is said to " love" the light sourcesinceit will stay thereindefinitely while the vehicle , on the right exploresthe world, liking to be nearits currentattractor but alwayson the , lookout for something . else to ( D Vehicle 4: By adding variousnonlinearspeeddependencies Vehicle 3, where ) the speedpeaks somewhere betweenthe maximum and minimum intensities other , . interestingmotor behaviorscan be observedThis can result in oscillatory navigation between two different light sources traced (top) or by circular or otherunusualpatterns arounda single source( bottom. ) from Braitenberg1984 Reprintedwith permission) . . ( Figures

Whence Behavior ?

,, (;Y - - - - - - ...\,. I""' ,' I /-~ .....4 ""/' /' / ......... -, , '',II-, -"5"( --- - - - - - - .." ,'" @ II'~
........ /' ....... """, ~ ' ~ , , \ \

/ I I , , \ \ \

/ /'

/ . \

\II '.:\"\\ -1\:: J ~/ 11

/

- "

\ \

\ \
I / / / / / .......

\ I I

'"
) (D ) Figure1.6 (continued

/ / .

'

ChapterI

attributedwith personalitytraits suchaspersistenceconsistencyinhumanity , , , or a frantic or observant nature Granted it is quite a leapto attributethese . traits , to robotsbuilt from suchextremelysimplecircuits and plastic toy blocks, but the merefact that an observer perceive can thesequalities evenmildly, in such , creatures notable is . simple

1.2.2 Artificial Intelligence

The birth of artificial intelligence(AI ) as a distinct field is generallyassociated with the DartmouthSummerResearch Conference held in August 1955 . 's This conference goalsinvolvedthe studyof a wide rangeof topics including use , language , neuralnets complexity theory self-improvement abstractions , , , and creativity In the original proposal( McCarthyet al. 1955 Marvin Min . ), " sky indicatesthat an intelligent machine would tend to build up within itself an abstractmodel of the environmentin which it is placed If it were given a . it could first explore solutionswithin the internal abstractmodel of problem the environment then attemptexternalexperiments This approach and ." dominated roboticsresearch the nextthirty years during which time AI research for , developeda strong dependence upon the use of representational knowledge and deliberativereasoningmethodsfor robotic planning Hierarchicalorganization . for planningwasalsomainstreamA plan is any hierarchicalprocess : in the organismthat can control the order in which a sequence operationsis of . , , performed ( Miller, Galanter andPribram 1960 p.16 . ) Someof the betterknown examples the AI planningtradition include: of . STRIPS This theoremproving systemusedfirst-order logic to developa : navigationalplan ( FikesandNilsson 1971 . ) . ABSTRIPS This refinementof the STRIPS systemused a hierarchy of : abstraction to spaces improvethe efficiencyof a STRIPStypeplanner refining , the detailsof a plan asthey becomeimportant(Sacerdoti1974 . ) . HACKER: This systemsearch througha library of procedures propose es to a . Jomain plan, which it later debugsThe blocks world < (toy blocksmovedabout by a simulatedoversimplifiedrobotic arm) servedasa primary demonstration venue(Sussman 1975 . ) . NOAH: This hierarchicalrobotic assembly plannerusesproblemdecomposition and then criticizes the potentially interactingsubproblemsreordering , their plannedexecutionasnecessarySacerdoti1975 . ( ) The classical AI methodologyhas two important characteristics( Boden 1995 : the ability to representhierarchical structureby abstractionand the )

~ WhenceRehavinr

" use of " strong knowledgethat employs explicit symbolic representational assertions aboutthe world. AI ' s influenceon roboticsup to this point wasin the idea that knowledgeand knowledgerepresentation centralto intelligence are , and that robotics was no exception Perhaps . this was a consequence AI ' s of . preoccupationwith humm-level intelligence Consideringlower life forms seemed . uninteresting Behaviorbasedrobotics systemsreactedagainstthesetraditions Perhaps . " Brooks ( 1987a said it best Planningis just a way of avoiding figuring out : ) what to do next." Although initially resisted as paradigmshifts often are the , , ' notion of sensingand acting within the environmentstartedto take preeminence in AI -relatedrobotics researchover the previousfocus on knowledge and . in hardware representation planning Enablingadvances robotic andsensor ' roboticscommunitys hypotheses ,had madeit feasibleto testthe behaviorbased . The resultscaptured imaginationof AI researchers the aroundthe world. The inception and growth of distributed artificial intelligence (OAI) paralleled thesedevelopmentsBeginning as early as the Pandemonium . system (SelfridgeandNeisser1960 the notion beganto takeroot that multiple competing ), or cooperatingprocess (referredto initially as demonsand later as es are capableof generatingcoherentbehavior Early blackboardbased . ) agents II suchasHearsay ( Erman al. 1980 referred et ) speech understanding systems to theseasynchronousindependent , , agentsas knowledgesources communicating with each other through a global data structurecalled a blackboard . 's of Mind Theory ( Minsky 1986 forwardedmultiagentsystems ) Minsky Society as the basisfor all intelligence claiming that althougheachagentis as , and interactionbetween simpleasit canbe, throughthe coordinated concerted thesesimple agents highly complex intelligencecan emerge Individual behaviors . , canoften be viewedasindependent in behaviorbased robotics , agents it closely to OAI. relating
1.2. 3 Robotics

Mainstreamroboticistshaveby necessity with generallybeenmoreconcerned and action than their classicalartificial intelligencecounterparts . perception To conductrobotics researchrobots are needed Thosewho only work with . , simulationsoften ignore this seeminglyobvious point. Robots can be complex to build and difficult to maintain To position currentresearch . relativeto ' them, it is worth briefly reviewing someof roboticists earliestefforts, bearing constrained these in mind that technologyin the 1960sand 1970sseverely . today projectscomparedto the computationalluxuries afforded researchers

Chapter1

Figure1.7 . (Photograph . Shakey courtesyof SRI International) Many other robots will be discussed throughoutthis book, but thesesystems arenotableaspioneers thosethat followed. for . Shakey: Oneof the first mobilerobots Shakey(figure 1.7), wasconstructed , in the late 1960s the StanfordResearch at Institute( Nilsson1969 . It inhabited ) an artificial world, an office areawith objects specially colored and shaped to assistit in recognizingan obj.ectusing vision. It plannedan action suchas , pushingthe recognizedobject from one place to another and then executed the plan. The STRIPSplanning systemmentionedearlier was developed for

Whence Behavior ?

usein this system The robot itself wasconstructed two independently . of con trolled steppermotors and had a vidicon televisioncameraand optical range finder mountedat the top. The camera motor-controlled tilt , focus, andiris had . were mountedat the peripheryof the capabilities Whiskerlike bump sensors robot for protection The plannerusedinformation storedwithin a symbolic . : world model to determinewhat actionsto take to achievethe robot' s goal at a given time. In this system perceptionprovidedthe information to maintain , andmodify the world model' s representations . . Hlli A RE: This project beganaround 1977 at Laboratoired' Automatique et d' Analysedes System ( LAAS) in Toulouse France(Giralt, Chatila and es , , Vaisset1984 . Therobot Hll. .ARE (figure 1.8) wasequipped with threewheels : ) two drive andone caster It wasratherheavy weighingin at 400 kg. Its world . ,

Figure1.8 Hn.ARE ( photograph . . of . , , courtesy LAAS CNRSToulouseFrance)

Chapter1

(A) Figure1.9 . contained smoothflat floorsfoundin a typical office environmentFor sensing the . , , , it useda videocamerafourteenultrasonicsensorsanda laserrangefinder : within a multilevel representational space Geometric Planningwasconducted of and the modelsrepresented actualdistances measurements the worlds, anda . the relationalmodelexpressed connectivityof roomsandcorridors Of special 's for experimentation . noteis Hll..AR E longevity. Therobot wasstill beingused after its initial construction( NoreilsandChatila 1989 . well over a decade ) . Stanford Cart/ CMU Rover: The StanfordCart (figure 1.9, A ) was aminimal robotic platform used by Moravec to test stereovision as a meansfor aboutonemeter 1977 . It wasquite slow, lurching ahead ) navigation( Moravec . , every ten to fifteen minutes with a full run lasting about five hours The vi -

Behavior ~ Whence

(B)
) Figure 1.9 (continued . courtesyof HansMoravecand the (A ) StanfordCart. (B) CMU Rover (Photographs . RoboticsInstitute CarnegieMellon University) ,

was sualprocessing the mosttime-consumingaspect but the cart success , fully obstacles , twenty metercoursesavoidingvisually detected fairly complex navigated were addedto its internal world map as detected as it went. Obstacles . as andwererepresented enclosingspheresThe cart useda graphsearch algorithm . to find the shortest paththroughthis abstractmodel Around 1980 Moravecleft for CarnegieMellon University (CMU) where , he led the effort in constructingthe CMU Rover (Moravec 1983 a smaller , ), robot with three independentlypoweredand steeredwheel pairs cylindrical / capableof carrying a cameramountedon a pan tilt mechanismas well as infrared and ultrasonicsensors (figure 1.9, B). This robot was followed by a

Chapter1

1.10 Figure
Robotcontrol systemspectrum . long succession of other CMU robots , several of which are described in other portions of this book . These and other robotic precursors set the stage for the advancesand controversies to come as behavior -based robotic systems appeared in the mid - 1980s.

1.3 TIlE SPECTRUM ROBOT OF CONTROL

and es Many differenttechniques approach for robotic control havebeendeveloped . Figure 1.10 depictsa spectrumof currentrobot control strategiesThe . left side represents methodsthat employ deliberativereasoningand the right reactivecontrol. A robot employingdeliberative represents reasoning requires relatively completeknowledgeabout the world and usesthis knowledgeto it , predictthe outcomeof its actions an ability that enables to optimizeits performance oftenrequires relativeto its modelof the world. Deliberatereasoning aboutthis world model primarily that the knowledge , upon strongassumptions is is . which reasoning based consistentreliable, andcertain If the information , usesis inaccurate haschanged or sinceobtained the outcomeof the reasoner , . reasoning mayerr seriously In a dynamicworld, whereobjectsmaybe moving arbitrarily (e.g., in a battlefieldor a crowdedcorridor), it is potentiallydangerous to rely on pastinformation that may no longer be valid. Representational from both prior knowledge world modelsare thereforegenerallyconstructed datain supportof deliberation . and aboutthe environment incomingsensor

? WhenceBehavior

Level 7 Levele Level 5 Level4 Level3 Level 2 Level 1

1.11 Figure '

. Albus s hierarchicalintelligent control system Deliberative reasoning systems often have several common characteristics: . They are hierarchical in structure with a clearly identifiable subdivision of es functionality , similar to the organization of commercial business or military command. . Communication and control occurs in a predictable and predetermined manner , flowing up and down the hierarchy , with little if any lateral movement. . Higher levels in the hierarchy provide subgoals for lower subordinate levels. . Planning scope, both spatial and temporal , changes during descent in the hierarchy . Time requirements are shorter and spatial considerations are more local at the lower levels. . They rely heavily on symbolic representation world models. 1. 3.1 Deliberative / Hierarchicai Control

thoseof reactive The intelligent control roboticscommunity whoseroots precede , -based as methods its principal behavior , reasoning systemsusesdeliberative and . , paradigm Albus, at the National Institute of Standards Technology ' ' . is one of this philosophys leading proponentsHis methodsattemptto integrate both natural and artificial reasoning(Albus 1991 . Figure 1.11 depicts )

Chapter1

a jukebox-like hierarchicalmodel with eachlayer consistingof four components , asoutlined in Albus' s theoryof intelligence sensory : ; , processingworld task decompositionand valuejudgment All layersarejoined by a . , , modeling . global memorythroughwhich representational knowledgeis shared Perhaps the most telling assertionthat represents heavyrelianceon world models the is reflectedin Albus' s views regardingthe role of perception Perception the : is establishment maintenance correspondence and of betweenthe internal world modelandthe externalreal world. Consequentlyactionresultsfrom reasoning , over the world model. Perception thus is not tied directly to action . In the mid- 1980s this view so dominatedrobotics that the government , a architecture reflectedthis model. Figure 1.12 shows that developed standard the NASA NIST( NBS standard / reference modelfor TelerobotControl System ) Architecture or NASREM (Albus, McCain, and Lumia 1987 . Despite the ) ' s endorsementNASREM hashad limited acceptance is but , government only still beingusedfor taskssuchascreatinga flight teleroboticservicercapable of and tasksfor NA SA' s spacestation performingmaintenance simpleassembly Freedom(Lumia 1994 . The six levelsembodiedon this systemeachcapture ) a specificfunctionality. Simply put, from the lowestlevel to the highest : 1. Servo providesservocontrol (position velocity, and force management : , ) for all the robot' s actuators . 2. Primitive: determines motion primitives to generate smoothtrajectories . 3. Elementalmove definesandplansfor therobot pathsfreeof collisionswith : environmental obstacles . 4. Task converts : desiredactionson a singleobjectin the world into sequences of elementalmovesthat canaccomplish them. 5. Service bay: convertsactions on groups of objects into tasks to be performed on individual objects scheduling taskswithin the servicebay area . , 6. Servicemission decomposes overall high-level missionplan into service : the . bay commands for . Higher levelsin the hierarchycreatesubgoals lower levels Anotherarchitecturalembodiment thesesameideas RCS (the Real time of , Control Systemreferencemodel architecture, hasthe samebasiclayering as ) NASREM but morefaithfully embeds components the outlinedin Albus' s theory of intelligence This approach . wastestedin simulationfor an autonomous submarine Huang 1996 but hasnot yet beenfieldedon the actualvehicle. ( ) In other work alongthesesamelines, researchers Drexel University have at focusedon the theory of intelligent hierarchicalcontrol and createda control modelpossessing following characteristics the :

Whence Behavior ?

SENSORY PROCESSING

WORLD MODELING

TASK DE MPOSmON CO

SENSE
figure 1.12 . architecture NASREM
. . .

ACT

It correlates human teams and robotic control structures: ' A hierarchy of decision makers implements this idea. Autonomous control systems are organized as teams of decision makers. . It assumesthat the task is decomposable, that is , it can result in structured subtasks. . Hierarchies are generated by recursion using a generalized controller . . Preconditions are established at each level of recursion to ensure proper execution.

Chapter1

. Figure 1.13 showsa mobile robot control systemconsistingof six levels The setof nested hierarchicalcontrollersconsists a high-levelplanner navigator of , , . , pilot, path monitor, controller andlow-level control system In yet anotherrepresentative the intelligent controlscommunity research of , at the Rensselaer Institute ( Lefebvreand Saridis 1992 restricted ) PolYtechnic the hierarchyto three primary levels organizationlevel (conductshigh-level : ) planning and reasoning, coordinationlevel (providesintegrationacrossvarious hardwaresubsystems and executionlevel (supportsbasic control and ), hardware. This approachimplementsthe principle of increasingprecision ) with decreasing . intelligenceasonedescends throughthehierarchyFigure 1.14 . depictsa logical model of this architecturalframework Note the clear and restrictive- flow of control and communicationbetweenlevels within the . hierarchy Hierarchicalcontrol is seeminglywell suitedfor structuredandhighly predictable environmentse.g., manufacturing. Reactivesystemshowever were ( , , ) in response severalof the apparentdrawbacksassociated to with developed the hierarchicaldesignparadigmincluding a perceived lack of responsiveness in unstructured and uncertainenvironments due both to the requirements of world modeling and the limited communication ; pathways and the difficulty in engineering completesystemsas incrementalcompetency proveddifficult to achieve that is, virtually the entire systemneeded be built beforetesting to , wasfeasible .

1.3.2 Reactive Systems

The right side of the spectrum reactivesystems depictedin figure 1.10 represents . Simplyput, reactivecontrol is a technique tightly couplingperception for and action, typically in the contextof motor behaviors to producetimely , robotic response dynamicand unstructured in worlds. We further definethe following: . An individual behavior a stimulusresponse : / pair for a given environmental n. settingthat is modulated attentionanddetennined intentio, by by . Attention prioritizes tasksandfocusessensory : resources is detennined and context . by the currentenvironmental . Intention: determines which set of behaviorsshouldbe activebasedon the ' robotic agents internal goalsandobjectives . . Overt or emergent : behavior the global behaviorof the robot or organismas a consequence the interactionof the activeindividual behaviors of .

? WhenceBehavior

Figure 1.13 . Nestedhierarchicalintelligent controller

Chapter1

Figure 1.14 J modelof hierarchicalintelligent robot. Logica . Reflexive behavior (alternatively , purely reactive behavior ) : behavior that is generated by hardwired reactive behaviors with tight sensor effector arcs, where sensory information is not persistent and no world models are used whatsoever. Several key aspects of this behavior-based methodology include (Brooks 1991b) : . Situatedness: The robot is an entity situated and surrounded by the real world . It does not operate upon abstract representations of reality , but rather reality itself . . Embodiment : A robot has a physical presence (a body ) . This spatial reality has consequences in its dynamic interactions with the world that cannot be simulated faithfully .

? WhenceBehavior

. EmergenceIntelligencearises : from theinteractions therobotic agentwith of its environment It is not a propertyof either the agentor the environmentin . isolationbut is rathera result of the interplaybetweenthem. This book focuses reactiverobotic systemswhose , prim~ ly on behaviorbased 4 describein moredetail. Hierarchi structureandorganization 3 chapters and further in the contextof hybrid robotic cal control, however is also discussed , in architectures presented chapter6.

RELATED ISSUES
and the A few importantissues centralto understanding appreciating behavior beforeheadinginto the core of basedparadigmwarrantadditionaldiscussion , this book. . Groundingin reality: A chroniccriticism of traditionalartificial intelligence researchis that it suffers from the symbol grounding problem that is, the , with which the systemreasonsoften have no physical correlation symbols . with reality; they are not groundedby perceptualor motor acts In a sense can : ungrounded systems be saidto be delusional Their world is an artifactual . hallucination Robotic simulationsare often the most insidious examplesof " this problem with " robots purporting to be sensingand acting but instead , to just creatingnew symbolsfrom old, noneof which truly corresponds actual . events Embodiment as statedearlier forces a robot to function within its , , of : environment sensing acting, and sufferingdirectly from the consequences , ." and its misperceptions misconceptions Building robotsthat aresituatedin the " world crystallizesthe hard issues (Flynn and Brooks 1989 . For that reason ) in this book focusesprimarily on real robotic systems implemented hardware for asexemplars robotic control. . Ecological dynamics A physical agentdoesnot residein a vacuumbut is : immersedin a highly dynamicenvironmentthat varies significantly typically in both spaceand time. Further theseenvironmentaldynamics except for , , , highly structuredworkplaces are very difficult if not impossibleto characterize . Nonethelessif a situatedrobotic agentis to be designed , , properly it must and within its designtheopportunities perils thatthe environment acknowledge es . affordsit . This is mucheasiersaidthandone In nature evolutionaryprocess , shapeagentsto fit their ecologicalniche; thesetime scalesunfortunatelyare not availableto the practicingroboticist. Adaptation howevercanbe crucially , ; chapter8 exploresthis further. important

Chapter1 . Scalability : Scalability of the behavior-based approach has been a major question from its inception . Although these methods are clearly well suited for low -level tasks requiring the competence of creatures such as insects, it has

beenunclearwhethertheywould scaleto conformto humanlevel intelligence . " the strict behaviorist Tsotsos( 1995 for example argues that for the , ), position modeling of intelligencedoesnot scaleto humanlike problemsand performance ." Section7.1 considers point further. Many of the strict behaviorists this has , persistin their view that the approach no limits ; notably Brooks ( 1990b ) statesthat " we believe that in principle we haveuncoveredthe fundamental foundationof intelligence" Othersadvocate hybrid approach . a betweensymbolic andbehavioralmethodsarguingthat thesetwo approach are es , reasoning :" hierarchical control fully compatible The falsedichotomythatexistsbetween " and reactivesystems shouldbe dropped (Arkin 1989d. (Seealsochapter6.) ) Much currentresearch focuses testingthe limits of behaviorbased on methods , andthis themewill recur throughoutthis book.

'S 1.5 WHAT AHEAD

This book consistsof the following chapters : 1. Introduction : highlightsthe core issues intelligent roboticsandreviews of the history of cyberneticsartificial intelligence androboticsthat led up to the , , of behaviorbased robotic systems . development 2. Animal behavior: studiesthe basis for intelligence biological systems , , , , throughthe eyesof psychologistsneuroscientistsand ethologistsand examines several robotic systems . representative inspiredby animalbehavior 3. Robot behavior: describes basisfor behaviorbased the robotics including , the notation expressionencoding assemblingandcoordinationof behaviors . , , , , -based architectures: presentsa range of robotic architectures 4. Behavior . employingthe behaviorbased paradigm 5. Representationalissuesfor behavioral systems questionsand explores : the role of representational within the contextof a behaviorbased knowledge . system 6. Hybrid deUberativeireactive architectures: evaluatesrobotic architectures that couplemore traditional artificial intelligenceplanningsystems with reactivecontrol systems an effort to extendfurther the utility of behavior in based control. 7. Perceptual basis for behavior-based control: considersthe issuesconcerning the connection perception action sensor of to mod, types perceptual

Whence Behavior ? utes expectationsattention and so on- and presents , , , perceptual designfor a . of specificapplications , rangeof robotic tasks including descriptions es 8. Adaptive behavior: address how robotscancopewith a changingworld mechanisms , including reinforcement througha variety of learningandadaptation . neuralnet Worksfuzzy logic, evolutionarymethodsandothers , , , learning of 9. Social behavior: opensup behaviorbasedroboticsto the consideration how teamsand societiesof robots can function togethereffectively raising new issuessuchas communicationinterference and multiagentcompetition , , , a , andlearning andpresents casestudyillustratingmanyof these cooperation
concepts. 10. Open issues: explores some open questions and philosophical issues regarding intelligence within artificial systems in general and behavior based robots in particular .

Chapter 2
Animal Behavior

Animals, in their generation are wiser than the sons of men; but their wisdom is , . , . confinedto a few particulars and lies in a very narrowcompass - Joseph Addison

Chapterofpossible Objectives between 1 develop .Torobot an animal understanding and .background control ,psychology behavior therelationships ethol 2fora . .To reasonable ,and provide in the of neuroscience roboticist systems ogya range robotic . 3 examine motivated .To wide biologically
2.1 WHAT DOES ANIMAL BEHAVIOR OFFER ROBOTICS? in The possibility of intelligent behavioris indicatedby its manifestation biological thenthat a suitablestartingpoint for the study . It seems systems logical of behaviorbasedrobotics should begin with an overview of biological behavior . . First, animal behaviordefinesintelligence Where intelligencebegins and endsis an open -endedquestion but we will concedein this text that intelligence , animals Our working definition will be that . canresidein subhuman endowsa system(biologicalor otherwise with the ability to improve ) intelligence to its likelihood of survival within the real world and whereappropriate success , fully with other agentsto do so. Second animal competeor cooperate

Chapter2

behaviorprovidesan existence . proof that intelligenceis achievableIt is not a a poorly understood , it is a concrete mysticalconcept reality, although phenomena . Third, the study of animal behaviorcan provide modelsthat a roboticist canoperationalize within a robotic system Thesemodelsmaybe implemented . with high fidelity to their animalcounterparts may serveonly asan inspiration or for the roboticsresearcher . ' Roboticistshavestruggled providetheir machines to with animals simplest : capabilities the ability to perceiveand act within the environmentin a mean . ingful and purposivemanner Although a studyof existingbiological systems that alreadypossess ability to conductthesetaskssuccess the obviously fully seems a reasonable methodto achieve that goal, the roboticscommunityhas it . historically resisted for two principal reasonsFirst, the underlyinghardware is fundamentallydifferent. Biological systems bring a large amountof evolutionary to supportintelligent behaviorin their siliconbaggage unnecessary basedcounterpartsSecond our knowledgeof the functioning of biological . , hardwareis often inadequate supportits migration from one systemto an to other For theseandother reasonsmanyroboticistsignore biological realities . , andseekpurely engineering solutions . Behaviorbasedroboticistsarguethat there is much that can be gainedfor roboticsthroughthe studyof neuroscience . , psychology andethology ,

The behaviorbasedroboticist needsto decidehow to use resultsfrom these otherdisciplines Somescientists . attemptto implementtheseresultsasclosely as possibly concerningthemselves , primarily with testing the underlying hypotheses of the biological modelsin question Otherschooseto abstractthe . underlying details and use thesemodelsfor inspiration to createmore intelligent robots unconcerned with any impact within the disciplinesfrom which , the original modelsarose Wewill seeexamples both approach within this . of es book. To appreciate behaviorbasedrobotics it is important to have somebackground , in biological behavior which this chapterattempts provide. We first to ,

Animal Behavior

of overviewthe importantconcepts neuroscience , psychologyandethology in , , . with several robotic systems whose that order The chapterconcludes exemplar , goalshavedrawn heavily on biological modelsfor robotic implementation a of themethat continues varying degrees to throughoutthe remainder the book.

VI I BASIS FORBEHA 0 R 2.2 NE ROSCffi NTI F C U

The centralnervoussystem(CNS) is a highly complexsubjectwhosediscussion warrantsat least a separate textbook This sectionattemptsonly a gross . . overview First, it highlights the componenttechnologyof neural circuitry. of Next, it introducesthe readerto the most basic aspects brain function and that translatestimulusinto restructureand the neurophysiological pathways models . abstract , computational . sponsethat is, producebehavior Last, it presents as within braintheorythathaveserved a basisfor behaviorbased developed robotic systems .

2.2.1

Neural Circuitry Eachoutcry of the huntedhare . A fibre from the brain doestear - William Blake

' is The nervoussystems elementalcellular component the neuron(figure 2.1). " " canonical neuron and : Theycomein manydifferentshapes Thereis no single . a sizes but they do possess commonstructure Emanatingfrom the cell body , es at the axonhillock is the axon which after a traversalof somelengthbranch , . off into a collection of synapticterminalsor bulbs This branchingis referred in . to as axonal arborealization The axon is often sheathed myelin, which of facilitatesthe transmission the neuralimpulsealongthe fiber. The boundary betweenneural interconnectionsreferredto as a synapseis wherechemical , , the diffuse across synapticcleft whenthe cell " fires." At the neurotransmitters ' from the cell body neurons receivingend a collection of dendritesemanates , , . from the other sideof the synapse continuing of the occursacross neuronby the conveyance an electrical Signaltransmission ' chargefrom the dendrites input surfacesthrough the cell body. If the total amountof electricity impinging upon the cell is below a certain threshold , , the currentis passivelypropagated throughthe cell up the axon becoming the es. weakeras it progress If , however it exceeds thresholdat the axon , without significant loss and hillock, a spike is generated actively propagated

2 Chapter
~
Dendri

8m ....

Axon Axonhitock
rborization I r . Axon

Figure2.1 of . (From Metaphorical The Brainby M. Arbib Copyright . Stylized representationa neuron 1972 WileyInterscience . Reprinted permission JohnWileyandSons of by , by Inc.)
of current up the axon to the synaptic bulbs , causing the release of neurotransmitters across the synaptic cleft . The cell must then wait a finite amount of time (the refractory period) before it can generate another electrical spike. This spike is also referred to as the action potential . Basic neurotransmitters are of two principal types : excitatory , adding to the probability of the receiving cell ' s firing ; and inhibitory , decreasing the likelihood of the receiving cell ' s firing . Combinations of neurons give rise to ever- increasing complex neural circuitry . There are many examples of specialized small systems of neurons whose function neuroscientists have elucidated. These special purpose systems include (from Arbib 1995b) : . . . . . . . . scratch reflex es in turtles bat sonar stomatogastric control in lobsters locomotion pattern generation in lampreys wiping reflex in frogs cockroach locomotion location of objects with electricity in electric fish visuomotor coordination in flies and frogs

Animal Behavior

Often roboticists can draw from these neural models to create similar fonD S of behavior in machines. In Section 2.5 , we will study a few examples of this

eachdedicatedto a spec function. Thesephysically parallel columnsalso "ific processinformation in parallel. This is of particular importancefor space from sensory , inputs. For example in touch, space mapsgenerated preserving ' s embeddedneural tactile sensors project preserving maps from the skin cortex This is also the case for visual input . to the brain' s somatosensory ' . from the eyes retinasthat ultimately projects onto the visual cortex Parallel of pathwaysare naturally presentfor the processing spatially distributed information. One model related to the inherentparallelism in neural processingis referred ' to aslateral inhibition, whereinhibition of a neurons firing arisesfrom . the activity of its neural neighbors Lateral inhibition can yield asingledom . inant pathway even when multiple concurrentactive pathwaysare present This results from amplification of the variationsin activity betweendifferent . neuronsor neuralpathways Throughstronglateral inhibition, one choice . from many can be selectedin a winner-take all manner This is of particular value in taskssuchas competitivelearningfor patternclassificationtasks , in of word sense language solving the correspondence , , prey recognition disambiguation problem in stereovision (finding matching featuresin two or moreseparate ), images or selectingonefrom amongmanypossiblebehavioral . responses

2.2.2 Brain StructureandFunction

' It is said that the Limbic systemof the brain controlsthe four F s: Feeding Fighting, , . , Fleeing andReproduction - Karl Pribram

. Animal brains obviously comein a very wide rangeof sizes Simple invertebrates - 1Q4neurons whereas brain the havenervoussystems , consistingof 103 . suchasa mousecontainsapproximately107neuronsThe of a smallvertebrate . individual neurons Despite to humanbrain hasbeenestimated contain 101- 1011 in brain size we can say severalthings generallyabout a largevariation , . . brains First, locality is a commonfeature Brains are not a homogeneous vertebrate into different massof neurons rather they are structurallyorganized ; , . Next, animalbrains eachof which containsspecialized , functionality regions

Chapter2

generallyhavethree major subdivisions(figure 2.2). For mammalianbrains , thesegenerallyconsistof ( 1) the forebrain which comprises the , . Neocortex associated : with higher level cognition. . Limbic system(bet ween neocortexand cerebrum: the ; ) providing basicbehavioral survivalresponses . . Thalamus mediating incoming sensoryinformation and : outgoing motor . responses . Hypothalamus managinghomeostasisthat is, maintaininga safe internal : , state(temperaturehunger respiration and the like). , , , the (2) the brainstem which comprises , . Midbrain: concerned with the processing incoming sensoryinformation of , (sight, sound touch, and so forth) and control of primitive motor response . systems . Hindbrain which consists the of , . Pons projectingacross brain : the . carryinginformationto the cortex . Cerebellum maintainingthe tone of muscle : for groups necessary coordinated motion. . Medulla oblongata connecting brain andthe : the spinalcord. and (3) the spinal cord, containingreflexive pathwaysfor control of various motor systemsFinally, afferent inputs conveysignals(typically sensory toward . ) the brain, whereasefferentsignalsconveycommands from the brain to the body. Invertebrate neuralstructureis highly variableandthusfewer gener alizationscanbe made . Mammaliancortex hasregionsassociated with specificsensoryinputs and motor commandoutputs (figure 2.3). In humans the visual cortex (sight) is , toward the rear of the brain, the auditory cortex (sound is to the side and , ) the somatosensory cortex (touch is midbrain, adjacentto the motor cortex ) from the input sensory (locomotion . Spacepreservingtopographicmappings ) within all theseregions It is interestingto note . organsto the cortexarepresent that thesemappingsare plastic in the sense that they can be reorganized after . This has been shownfor both the somatosensory damage system(Florence and Kaas 1995 andfor visual cortex(Kaaset al. 1990 . ) ) occurs within the brain as well. At this level neuroscientific Subspecialization modelshaveoften had an impact on behaviorbasedrobot design For . : example " " . In section7.2.2 we encounter what" and" where corticalregionsassociated with visual processing .

Animal Behavior

Brain Primitive Stem

::~ ; ~ ; ~~ db. forebrain mlraIn .

Brain Reptilian

~ :~:::. ~ ~
Brain Chimpanzee

RatBrain ) (notto scale

cerebra hemisp ~ml ~'tJ ~. Hforebra ~iraIn =~ hindbrain 'd) b

brain stem cerebellum

::~ml:2db ~ ~SZforebrain ~.: raIn

Mammalian Brain

Human Brain
left cerebral
here

Figure2.2 .) of . (Figure Animalbrainstructures courtesy RodGropen . Section6.2 discuss distinctionsbetweendeliberative(willed) and automatic es motor actionsbased for behavioralcontrol systems managing uponneu . rophysiologicalevidence . Evidencefor parallelmechanisms with both long- andshort term associated , ) (Miller andDesimone1994 . For both cases memoryhasalsobeenuncovered : onefor factsandevents are , two distinctprocessing present systems seemingly es skills. Section5.2 discuss therole the otherfor learningmotor andperceptual . robotic systems of thesetwo different typesof memoryfor behaviorbased

2 Chapter
somatosensory cortex
motor cortex

auditory

cortex
primary cortical secondary cortical association

Fiaure 2.3
Regions of sensory and motor process in the human cortex . The general flow of infor -

mationwithin brain indicaterlby arrows ( Figurecourtesyof Rod Grope ) the is . D.

for of Neurobiologyoften argues the hypothesis a vectorialbasisfor motor control, somethingthat can be readily ttanslatedinto robotic control systems at (section3.3.2). Research MIT ( Bizzi, MussaIvaldi, and Giszter 1991 has ) shown that a neural encodingof potential limb motion encompassing direction , amplitude and velocity existswithin the spinal cord of the deafferented , frog. Microstimulation of different regionsof the spinal cord generates specific force fields directingthe forelimb to specificlocations Theseconvergent . force fields move the limb towardsan equilibrium point specifiedby the region stimulated(figure 2.4) . The limb itself canbe considered set of tunable a springsasit movestowardsits restposition(equilibrium point). Thusthe planning of the aspects the CNS ttanslateinto establishing equilibrium points that a desiredmotion. Of particular interest is the observation implicitly specify
Figure 2.4 (A ) Forcefields generated microstimulationof lumbarregions(A -D) of frog spinal by cord (shownat left). of ( B) Superposition multiple stimuli. C denotes of simple vector summation independent fields A and B. D represents actualfield evokedby microstimulationof regionsA and B conc ' ently. (Reprintedwith permissionfrom Bizzi, MussaIvaldi, and Giszter Un . 1991 Copyright 1991by AmericanAssociationfor the Advancement Science of .)

Animal Behavior

A .,"" It '.~t',," \-,~ ,." ,'" '" "" ~ ,'.'\" , " /'
.......... ..... .......... ....... , I - . ---. -- _ . . - . ~ . .. ~ . . .

ID ,, ~J ~~ ' ...'\\I;
. ' ' " " \ \ \ ' ' \ \ \ \ \ \ \ ~ , .
.

(A)

IVif ~" ~ " ,~ ,.,

" I' ~ ; ' '

"t""~ """"" ,,,',\'~ t~~ ,\' , :~ ,:~

, . . . , . . .

"

"

"

"

"

"

"

" " " " " " ... -- " - ~ .. ,

-

" 111 " 11 " 11 ' & 1 " , & II \ " I ' I ' . . , . ..
. .

.--_.'~ ---.,'"-~"'" .,~ .,I\~ ~ """"

" , I ' ,-" ,

\ \ , I , ~ ,. " ' \ \ I / ' ~ ", I " ' \ \ 1 " -" ,,, ' " \ . , , ,. " ' " \ . " " -'

. " " " "

(B)

~,.','," ~", ", --",,.., --,. , -""

" " " " .1

\ I I I ' ~ ' \ I I I ' ~ ~ \ \ III " \ III ~ " \ 11 / -' "

2 Chapter Box2.1 Memory Types

over , " as by Information . that to STM . hours to temporary transfer actions in retention refer " viewed distinction acting the , process a memory is minutes " in inferior with to is requiring usually measured appropriate we is significant memory working tasks a " involved , is seconds what accuracy as guide LTM tenn information is to to for is appears several recall Long memory . of scale its performing ) LTM " infonnation memory . referred brain but tenn for time , 1981 information the tenn also LTM periods of of is The holding to involves . Short for : Long ) for capacity : ) definition that STM STM its region . , ( McFarland ) STM in ( persists LTM . " ( from scale 1995 capacity manipulation conversation working STM its hours the in time in limitless memory memory 24 One memory hippocampal . Bumod and everyday tenn information stored than The term and almost in term years limited . of as long or storage intennediate longer STM quite Short Long Guigon ( an is from memory many . days of of process .

that multiple stimulationsgive rise to new spatial equilibrium points generated -based . by simple vector addition This sameprinciple is usedin schema robotic control (section4.4). New experiments humans(intact fortunately in ), havebeenshownto be consistent with this force-field model when appliedto and reachingtasks(Shadmehr MussaIvaldi 1994 . ) Othercompellingexamples that computations motion within the of arguing brain should be consideredas vectorsinclude researchfrom the New York University Medical Center( pellioniszand ilinas 1980 . The authorscontend ) that activity within the brain is vectorial Intendedmotion is generated an . as -dimensional . Brain function is considered activity vectorwithin a three space " of ." geometricwherethe language the brain is vectorial The authorsexplain , however that simple reflex are not adequate explain the entire rangeof es to , . complexityevidenced the actionsthe brain generates by Another exampleforwarding spatial vectorsas an underlying representational mediumfor neuralspecification motorbehaviorcomesfrom theJohns of " Schoolof Medicine (Georgopoulos 1986 . The " vector hypothesis Hopkins ) asserts the changes the activity of specificpopulationsof neuronsgenerates that in a neural coding in the fonD of a spatial vector for primate reaching . resultshave been shown to be consistentwith this underlying Experimental . hypothesis has Finally, vector based trajectorygeneration servedasan accountfor certain ' fonDS of animal navigation Arbib and Houses model ( 1987 explains . )

Animal Behavior

detourbehaviorin toads(i .e., their circumnavigation obstaclesby describing of ) the animal' s path planningin tenD S the generation divergence of of fields ' directional vectors basedon the animal s perceivedenvironment Inparticular . ( ) , repulsivefields surroundingobstacles attractiveforces leading to food , sourcesanddirectionalvectorsbased the frog' s spatialorientationgenerate on , a computational model of path planningin toadsconsistent with observed experimental -based data This modelhasbeeninfluential in the designof schema . robot controllers(section3.3.2).

2.2.3

Abstract Neurosci enti:fi(: Models

' , Unfortunately our knowledgeof the brain s function is still largely superficial in is at (literally). Progress neuroscience proceeding a rapid paceas new tools . brain function becomeavailable Nonethelessit has been for understanding . , said that even if we possessed completeroad map of the brain' s neural a structure (all of its neuronsand their interconnections our understanding ), would still be inadequateBrain activity over the neural substrateis highly . and control would still need , dynamic and information regardingprocessing to be elaborated . What then should a brain theorist do? The key for many scientistslies in their first formulating an abstractionof brain function and then looking for . neuralconfirmation This top-down approach characterizes many researchers in neuroscienceand has potentially high payoff for roboticists as abstract , , modelsof brain function hypothesized theseneuroscientists potentially can by leadto robotic control systems useful in their own right. Abstract computationalmodels used to expressbrain behavior have two forms: schema . es mainstream theoryandneuralnetworks Thesetwo approach arefully compatible(figure2.5). Schema is a higher level abstraction theory by . which behaviorcan be expressed modularly Neural networksprovide a basis occursat a lower for modelingat a finer granularity whereparallelprocessing , level. Schematheory is currently more adept at expressingbrain function, . whereasneural networkscan more closely reflect brain structure Schemas , onceformulated may be translatedinto neural network modelsif desiredor , deemed . In necessary this book we studyboth methodsin the contextof what . they offer behaviorbasedrobotic control systems

. 2.2.3 Schema .1

: Methods

The use of schemas a philosophicalmodel for the explanationof behavior as . datesas far back as Immanuel Kant in the eighteenthcentury Schemas

Chapter2

Figure 2. 5 Abstract behavioral models. Schemas or neural networks by themselves can be used to represent overt agent behavior , or schemascan be used as a higher - level abstraction that is in turn decomposed into a collection of neural networks .

weredefinedas a meansby which understanding ableto categorize is sensory of . perceptionin the process realizingknowledgeor experienceNeurophysio . logical schema theory emerged early in the twentiethcentury The first application was an effort to explainposturalcontrol mechanisms humans( Head in and Holmes 1911 . Schematheory has influencedpsychologyas well, serving ) asa bridging abstraction between brain andmind. Work by Bartlett ( 1932 ) and Piaget( 1971 usedschematheory as a mechanism expressing for models ) of memory and learning Neisser( 1976 presented cognitive model of . a ) interactionbetweenmotor behaviors the form of schemas in interlocking with in the contextof the perceptual . Normanand Shallice( 1986 perception ) cycle usedschemas a meansfor differentiatingbetweentwo classes behavior as of , willed and automatic and proposeda cognitive model that usescontention , as and competitionbetween schedulingmechanisms a meansfor cooperation behaviors Sections6.2 and 7.2.3 discusstheselast two examples . further. Arbib ( 1981 wasthe first to considerthe applications schema of ) theoryto robotic

Neural

"

function

Networks

avior structure \ ( Modeling Behavioral ) Models Schema functional ( decomposition

Animal Behavior

sion systems RisemanandHanson1987 . ( ) " " Many definitionsexist for the term schema, often strongly influencedby its applicationarea(e.g., computationalneuroscientificpsychological. Some , , ) : representative examples . a patternof action aswell asa pattern action ( Neisser .for 1976 ) . an adaptivecontroller that uses an identification procedureto updateits of ) representation the objectbeingcontrolled (Arbib 1981 . a functional unit receiving specialinformation, anticipatinga possibleperceptual content matchingitself to the perceivedinformation (Koy-Oberthur , 1989 ) . a perceptual whole corresponding a mentalentity ( Piaget1971 to ) Our working definition is as follows: A schemais the basic unit of behavior from which complex actions can be constructed it consistsof the ; of how to act or perceiveas well as the computational knowledge by process which it is enacted Schematheory providesa methodfor encodingrobotic . behaviorat a coarsergranularitythan neuralnetworkswhile retainingthe aspects of concurrent cooperativecompetitivecontrolcommonto neuroscientific models . Variousneurocomputational architectures have beencreatedthat incorporate theseideas For example work at the University of Genova(Morasso and . , of es ) Sanguineti1994 hasled to the development a model that encompass the vector basedmotion-planning strategies described earlier within the posterior . are into , parietalcortex Here multiple sensorimotor mappings integrated a unitary -orientedmovements for . , body schema necessary the generationof goal Vectorbased potentialfields (section3.3.2) providethe currencyof task specification for this integrationof taskintentions Later, section4.4 exploresother . modelsand associated methodsfor operationalizing schema theory as a basis -based control. for behavior

2.2.3.2 Neural Networks

, Computationalmodelsfor neural networks also referredto as connectionist of , systemshavea rich historyparallelingthe development traditionalsymbolic AI . Someof the earliestwork in the areacanbe tracedto the McCulloch-Pitts model of neurons( 1943 . McCulloch and Pitts used a simple linear threshold ) to unit with synapticweightsassociated eachsynapticinput. If a threshold . was exceededthe neuronfired, carrying its output to the next neuron This ,

Chapter2 synaptic weights

Xl w1 --" X----~--"'~ 2- -~-. -"---- :~X- -!3_ ~ .3 - -_ -: . . Xn Input ( Vector ) b2 inary 6A of is each Figure inputs compone 's a A .ronresult by then . vectormultiplied assoc percept Xitogether binary (:)and to .The 's output is binary then then Wito unit subjec synapticsummedis weight the 1 .This ) thresholding output operation sentcell . on indetermine to (the the8network next
Output

simplemodel gaverise to networkscapableof learningsimple patternrecognition tasks Rosenblatt( 1958 later introduceda formal neural model called . ) a percept (figure 2.6) andthe associated ron ron percept convergence proof that established of thesenetworksystemsIn the 1960s . provablelearningproperties and 1970s however neural network researchwent into a decline for avariety , , of reasons including the publication of the book Perceptrons Minsky , ( andPapert1969 which provedthe limitations of single layer percept networks ron ), . In the 1980s however the field resurgedwith the advent of multilayer , , neural networks and the use of backpropagation Rumelhart Hinton, and , ( Williams 1986 as a meansfor training these systems Many other notable . ) efforts within connectionism , during the last decade far too numerousto review here have yielded highly significantresults It shouldbe remembered . , , however that mostneuralnetworksareonly inspiredby actualbiological neurons , and provide poor fidelity regardingbrain function. Nonethelessthese , abstract modelshaverelevance the behaviorbased to roboticist computational andhavebeenusedwidely in tasksrangingfrom visual road following strategies section7.6.1) to adaptationand learning in behavioralcontrol systems ( 1989provides a more generaltreatmentof neural (section 8.4). Wasserman . networks

Animal Behavior

2.3 PSYCHOLOGICAL BASIS FORBEHAVIOR

is is , Psychology brainless neuroscience mind1e~~ - Les Karlovits

has Psychology traditionallyfocusedon the functioningof the mind, lesssothe brain. It is not our intent to revisit the classicalmonistdualist debateof mind / ' andbrain herebut ratherto look at what a psychologists perspective offer can
robotics . Certainly psychology is preoccupied with behavior. Within that scope, we focus on perception and action , as these issues are of primary concern for the roboticist , and provide a brief history of the field , beginning with the twentieth century. Sensory psychophysics was the first to relate stimulationintensityto perception

. Weberand Fechnerdeveloped the physicallaws that described relationships ' betweena stimuluss physicalintensity and its intensity asperceived by an observer Pani 1996 . ( ) Behaviorismburst upon psychologyin the early 1910s Behavioristsdiscarded . all mentalisticconcepts sensationperception image desire purpose : , , , , , 1925 . Behaviorwasdefinedby , andemotion amongothers( Watson , ) thinking observation only; datawas obtainedfrom observingwhat an organismdid or 's said. Everythingwas castin termsof stimulusand responseThis approach . main benefitwas making the field more scientificallyobjective moving away , from the use of introspectionas the primary basisfor the study of mind. Its mainclaim was" that thereis a response everyeffectivestimulusandthatresponse to " is immediate ( Watson 1925 . As behaviorism , ) progressedpsychology moreandmorescientificandlessphilosophical sociological asa field became , , andtheologicalby relying heavily uponempiricaldata(Hull 1943 . B.F. Skin) ' ner ( 1974 eventuallybecame behaviorisms bestknown proponent . ) Kohler 1947 brought physics into the fray, drawing Gestaltpsychology( ) ' from the tradition of sensorypsychophysics while broadening behaviorisms basis This form of psychologyinvertedbehaviorism . somewhatconcerning itself , with sensory visual) andhow behaviorarises heavily input (predominantly of asa direct consequence the structure the physicalenvironment of interacting " wasderivedfrom the Germanwhereit " with the agentitself. The term gestalt . enabled certainbehaviors referredto form or shape anattribute Certaingestalts as based the physicsof retinal projectionandthe ability of the perceiver upon to organizethe incoming stimuli. Gestaltpsychologyfocusedon perception

Chapter2

whereasbehaviorismprincipally concerned itself with action ( Neumann and Prinz 1990 . Gestalters however felt that behaviorismwas limited, arguing , , ) that levelsof organizationexist abovethe sensation itself, which an organism could useto its advantage . a , ), Ecological psychology as advocated J. J. Gibson ( 1979 demanded by of the environmentin which the organismwas situated deepunderstanding and how evolution affectedits developmentThe notion of affordances discussed . ( ' further in section7.2. 3) providesa meansfor explainingperceptions roots in behavior This psychologicaltheory saysthat things are perceivedin . termsof the opportunitiesthey afford an agentto act. All actionsare a direct of consequence sensorypickup. This resultsfrom the tuning by evolution of an organismsituatedin the world to its availablestimuli. Significantassertions (Gibson 1979 include: ) . The environmentis what organismsperceive The physical world differs . from the environmentthat is, it is morethan the world described physics . , by . The observer the environment and eachother . complement . Perceptionof surfacesis a powerful meansof understanding environment the . ' . Information is inherentin the ambientlight and is picked up by the agents . optic array Later, cognitive psychologyemerged paralleling the adventof computer , science defining cognition as "the activity of knowing: the acquisition organization , , " and , and use of knowledge ( Neisser1976 . Information processing ) modelsof the mind beganto play an ever increasingrole. Behaviorism computational wasrelegated the role of explaininganimalbehaviorandbecame to far less influential in studying humanintelligence Unifying methodsof explaining . the relationshipbetweenaction and perception(section7.2. 3) were ) developedunder the banner of cognitive psychology( Neisser1976 . Mentalistic terms previously abandoned could now be consideredusing compu tational process or metaphorsSomeof the underlying assumptions the es . of information processing ( ) approach Eysenck1993 include . A seriesof subsystems es information (e.g., stimulus process environmental ~ attention~ perception~ thoughtprocess ~ decision~ response es ). . The individual subsystems transformthe datasystematically . . Informationprocessing peoplestronglycorrelates in with that in computers . . Bottom-up processing initiated by stimuli, top- down processing intentions is by andexpectationssection7.5.4). (

Animal Behavior

Connectionism the associated and of development neuralnetwork technology (seeSection2.2.3) offer anotheralternativecomputationalmodel to explain mentalprocessingavailableto be exploitedby psychologists . , hasfluctuatedsignificantly depending the current on Although psychology , school of thought roboticistscan derive considerable benefitfrom an understanding , of thesedifferent perspectivesThe roboticist' s goals are generally . different: Machineintelligencedoesnot necessarily explanation requirea satisfactory of humanlevel intelligence Indeed evenpasse . theories , psychological canbe of valueasinspirationin building behaviorbasedautomatons .

LOGICAL 2.4 Emo ~ BASIS FORBEHAVIOR

are characters a kindof old sagastylizedbecause themost in . Animals stylized even acute themhave of little leeway theyplayouttheirparts as . - Edward Hoagland . Ed1ologyis die study of animal behaviorin its natural environment To die ' strict ed1ologistbehavioralstudiesmust be undertaken die wild ; animals in , haveno meaningoutsided1eirnatural setting The animal itself is . responses of , only onecomponent die overallsystem which mustincludedie environment in which it resides . Konrad Lorenz and Niko Tinbergen are widely acknowledgedas die foundersof die field. Tinbergen considered studiesto focuson four ed1ological areasof behavior( McFarland 1981 : causationsurvivalvalue development , ) , primary . into , andevolution Animal behavioritself canbe roughly categorized threemajor classesBeer Chiel, andSterling 1990 McFarland1981 : , , ( ) . Reftexesare rapid, automaticinvoluntary responses triggeredby a certain environmental stimuli. The reflexiveresponse persistsonly as long as die duration of die stimulus Further die response . wid1die stim, intensitycorrelates 's ulus strength Reflex areusedfor locomotionandod1er . es highly coordinated activities Certainescape . behaviors suchas d1ose found in snails and bristle , worms involvereflexiveactionthatresultsin rapidcontraction specificmuscles of , relatedto the flight response . . Taxesare behavioralresponses orient die animal towardor away from d1at a stimulus (attractiveor aversive. Taxesoccur in response visual, chemical to ) in . , mechanical and electromagnetic , phenomena a wide rangeof animals is Chemotaxis evidentin response chemicalstimuli asfound in die ttail following to of ants Klinotaxis occursin fly maggotsmoving towarda light source .

Chapter2

, by comparingthe intensityof the light from eachsideof their bodies resulting in a wavy course Tropotaxisexhibitedby wood lousesresultsin their heading . . directly towardsa light sourcethroughthe useof their compound eyes . Fixed-action patterns are time-extendedresponse patternsbiggered by a stimulus but persistingfor longer than the stimulus itself. The intensity and durationof the response not governed the strengthand durationof the are by stimulus unlike a reflexivebehavior Fixed-action patternsmay be motivated . , , unlike reflex , andthey may resultfrom a muchbroaderrangeof stimuli than es thosethat govern a simple reflex. Examplesinclude egg rebieving behavior of the greyling goose the songof crickets locust flight patterns and crayfish , , , . escape Motivated behaviorsare governednot only by environmentalstimuli but also by the internal stateof the animal being influencedby such things as , . appetite as Ethologistssuchas Lorenz adoptedthe notion of schema well. Schemas of es in , capturecomplicatedcombinations reflex , taxes and patternsreleased to a suitablecombinationof stimuli. A sign stimulusis the particular response external stimulus that releasesthe stereotypicalresponse Schemas which . , werelater renamed innate releasingmechanismsI RMs in an effort to clarify ( ) their meaning( Lorenz1981 havethe following traits (LorenzandLeyhausen ), 1973 : ) . An IRM is a simplified renderingof a combinationof stimuli eliciting a in . , , specific perhaps complex response a particularbiological situation . One IRM belongsto one reactionto a given situation attunedto relatively , few distinctivefeaturesof the environment oblivious to the rest. and . Every action dependent its own releasingschema on may be elicited completely of independently all otherreactionsintendedfor the sameobject. . The innate releasingmechanism providesthe overall meansfor a specific stimulusto release stereotypical a within a given environmental sign response context . For Lorenz and Tinbergen(Lorenz 1981 complex systemsof behavioral ), mechanisms had a hierarchicalcomponent although Tmbergenconsidered , this a weak commitmentuseful principally only for organizational . purposes Figure 2.7 shows an examplefor the display behavior sticklebackfish use in protecting territory. This notion of hierarchicalgrouping has parallels in serveas aggregates component of schema theory as well, whereassemblages schemassection3.4.4). (

" ". 0 1 ,
) ' 01 6 ' ~ ~ ~ . O11 ' ~ O e ' 1 , .

'll,QiJ iPI -rr-" "e6 '(ch .'A ,"6, r:Q vqr CP ~I: QlJ o~o ~< ~-il'~ IqC ;.:peilo ..~ O ~ '

2 Chapter
Maximum Selecting System Sensor

Sensor

Sensor

Sensor

Figure 2.8 . Model of maximum selectionsystemproviding lateral inhibition betweenbehaviors a a Whenever behavioris readyand its sensorystimuluspresent it generates positive , . . while inhibiting otherpotentiallyactivebehaviors After Lorenz 1981 response

[~ ~ [~

, Reciprocalinhibition of parallel behaviors a form of lateral inhibition, is also available (figure 2.8). In one model a maximumselecting systemen , and ablesoneof manybehaviorsto dominatebasedon its readiness incoming stimuli. This winner-take all strategyis commonwith many of the arbitration roboticssystems section3.4.3). All behaviors usedin behaviorbased methods ( . not active at a particular momentare inhibited centrally Supportingexperimental existsfor this locusof superior commandin animalsranging evidence to from invertebrates primates( Lorenz1981 . Hybrid behaviorbasedrobotic ) in chapter6, exploit the utility of this organi at architecturesdiscussed length , . zationalconcept are studiesin animal communicationmechanisms highly relevant Ethological for multiagentrobotic systemsArkin andHobbs1992 . Displaybehavior , ) ( in in particular involves the signaling of information by changes postureor ,

Animal Behavior

and activity. Thesestereotyped often highly unusualdisplaysare most often -action patterns(Smith 1977 and may be visible, audible , ) by generated fixed tactile, chemical or evenelectrical as in the caseof the electric eel. The displays , , themselves includebirdsong raisingof a dog' s hackles courtingdisplays , , in ducks color changes fish, leg waving in spiders etc. Suchdisplayshave in , , behavior evolutionarybenefitsfor suchactivitiesasindirectly invoking escape in thepresence predatorsreducingthe likelihood of fighting, andfacilitating of , . themselves , may be behavioralselection mating amongmanyothers The messages that enablethe recipientto respondappropriatelyfor a given messages . situation (i .e., " what" to do) such as flee, in the caseof an alarm message " " , suchas who messages messages They may alsobe so-callednonbehavioral " for kin or sex recognition or " where messagesproviding location information still from the senderThesenonbehavioral . messages may ultimately affect ' s behavior but not so . Ethologists asbehavioral the recipient , directly messages for havedeveloped methods analyzingandrepresenting complexandritualistic . interactions suchascourtshipand greeting . One of the most important conceptsfor behaviorbasedroboticistsdrawn the field of ethology is the ecological niche. As definedby McFarland from " , , ( 1981 p. 411), The statusof an animal in its community in terms of its relationsto food and enemies is generallycalled its niche." Animals survive , in naturebecause ably stableniche: a place where they have found a reason this . can coexistwith their environment Gibson( 1979 strongly asserted ) they mutuality of animal and environmentas a tenet of his school of ecological psychology(seesection2.3). Evolution hasmoldedanimalsto fit their niche. Further as the environmentis always to some degreein flux , a successful , or to animal mustbe capableof adaptingto somedegree thesechanges it will in habitat climate, food asserted changes . , by perish Environmentalpressures ' . sourcesandthe like, canprofoundly influencethe species survivability , of . This conceptof niche is important to roboticistsbecause their goals If and that is autonomous cansuccess theroboticistintendsto build a system fully inhabitants that systemmust find a stable , competewith other environmental the . nicheor it (asan application will be unsuccessfulThis promulgates view ) with mustfind their placewithin the world ascompetitors that robotic systems other ecological counterparts e.g., people . For robots to be commonplace , ) ( / mustfind the ecologicalnichesthat allow themto surviveand or dominate they or their competitors whetherthey be mechanical biological. Often economic , . are pressures sufficientto preventthe fielding of a robotic system If humans taskasa robot (e.g., vacuuming at a lower cost arewilling to performthe same ) and or with greaterreliability, the robot will be unableto displacethe human /

Chapter2
worker from the niche he already occupies. Thus , for a roboticist to design effective real world systems, he must be able to characterize the environment effectively . The system must be targeted towards some niche. Often this implies a high degree of specialization . These same arguments are often used in economics and marketing and are generalizable to behavior-based robotics ( McFarland and Bosser 1993) . Section 4.5.7 presents one example of a nichebased robotic architecture.

2.5 REPRESENTATIVE EXAMPLF~ OFBIOROBOTS ~

Let us begin our discussion of biorobots by summarizing some important lessons animal behavior affords the roboticist : . Complex behaviors can be constructed from simpler ones (e. ., g through hierarchies or sequentially, as in fixed -action patterns) . . Perceptual strategies should be tuned to respond only to the specific environmental stimuli relevant for situation - specific responses . . Competing behaviors must be coordinated by selection, arbitration , or some other means. . Robotic behaviors should match their environment well , that is , fit aparticular ecological niche. We now turn to five representative examples of robotic systems heavily motivated by animal studies. The first two focus on perceptual aspects that , is , sensory devices mimicking chemotaxis in ants and the compound eye of the fly . A pair of examples then illustrates the problem of producing coordinated locomotion for a robotic cockroach and a primate swinging from trees. The last case concerns interagent communication for a robotic honeybee. These examples are but interesting pieces of the puzzle of building robots ; subsequent chapters explore complete behavior-based robot design. 2. 5.1 Ant Chemotaxis Go to the ant, thou sluggard considerher ways andbe wise. , ,
- Proverbs 6:6

Ant behavioris of keeninterestto roboticistsbecause arerelativelysimple ants creatures of complexactionsthroughtheir social behaviorand biologists capable havestudiedthemextensivelyExcellentreference . works areavailableon antbehavior(e.g., Holldobler andWIlson 1990 . Much animalresearch ) signif-

Animal Behavior J

: until icantly influencedmultiagentrobotic systems We defer that discussion ' 9. For the momentwe considerhow chemicalsensinginspiredby ants , chapter behavior canbe usedfor pathfollowing in robots . , Ant communicationis predominantlychemical Visited paths are marked . a volatile b ail pheromoneAll antstraversinga useful path continually ' . using add this odor to the trail, strengthening reinforcing it for future use The and . variationsin foraging strategies result in a wide rangeof speciesspecificcollective that of patterns haveevolvedto fit the ecologicalneeds the environment to which they are adaptedIt could be useful for one interested developing . in robots capableof foraging over long distances to considerthe models forwarded , . by ant entomologists Simulationstudiesconducted the University of Brusselshaveshownthe at ' spontaneous developmentof biologically plausible b ails using mathematical behavior models Intemest traffic for the Argentine ant has been simulated . ) using pheromonemodels (Aron et al. 1990 . Deneubourgand Goss ( 1989 have reproducedspeciesspecific foraging patternsfor three different ) . ) army ant species Goss et al. ( 1990 have likewise emulatedcomputation ally the rotation of foraging trails observedin the harvesterant. Thesesimulation studies although encouraging still require implementationon real , , robots to gain widespreadacceptance useful models for robot foraging as behavior . Researchers Australia havetaken a step forward towardsmore directly in ant behavior by creating robotic systemscapableof both laying emulating down and detectingchemicaltrails (Russell Thiel, and Mackay SiI D 1994. , ) exhibit chemotaxisdetectingandorientingthemselves : Thesesystems alonga chemicaltrail. Camphor a volatile chemicalusedin mothballs servesas the , , ' : chemicalscent The applicationmethodis sbaightforward the robot dragsa . felt-tipped pen containingcamphoracrossthe floor as it moves depositinga , is . trail.onecentimeter wide (figure2.9a). Sensing morecomplex The detection devicecontainstwo sensorheadsseparated 50 mm (figure 2.9b). An inlet by drawsin air from immediatelybelow the sensoracrossa gravimetricdetector crystal. An air downflow surroundingthe inlet insuresthat the inlet air is . arriving from directly below the sensor The detectorcrystal is treatedwith a coating that absorbscamphor and as massis added the crystal' s resonant , , . in proportionto the amountof camphorabsorbedWhen frequencychanges to this chemotactic hasbeenattached a trackedmobile robot provided system ' with an algorithm that strivesto keep the odor b ail betweenthe two sensor b inlets, the robot hasbeenable to follow the chemotactic ' ail success fully for ' up to one half hour after the applicationof the camphorb ail.

2 Chapter

Direction ofMotion
Odor Trail (A)

! ~~ ~

Ii CS 1 8~ m

I t\
FloorSurface (8) .9 Figure2 Chemotaxis hardwareA) thecamphor :( device . ;( applicator 8) thedualhead sensing

. ~ ~

Animal Behavior

Figure2.10 Robotequipped compound consistingof 100 with facets a : eye providing 360degree view . panoramic of thehorizon

2.5 Fly VISion .2

' Research Frances CentreNational de la Recherche at Scientifique(C.N.R.S.) ' has consideredthe houseflys compoundeye a useful way in which a robot ' can view the world ( FranceshiniPichon and Bianes 1992 . The houseflys , , ) visual navigationsystemconsistsof approximatelyone million neuronsthat , , constantlyadjustthe amplitude frequency and twist of the wings, which are controlled by seventeen different muscles Visual motion is used for course . control. The eye of the houseflyis composed 3,000 pixels eachcontaining of andoperatingin parallel. Several behaviorspecificvision eight photoreceptors havebeenreported including vision for sexualpursuit of matesand , systems the detectionof polarizedlight for usein navigation(Mazokhin Porshnyakov 1969 . ) An in- depth study by the CNRS group has led to the developmentof a reactivemobile robot that usesan insect like visual system(figure 2.10 . This ) ' s raison d' etre is system simpler than that of the fly : it merely is to move safelyaboutthe world, avoidingobstacles exploiting via vision the relative by motionbetween itself andtheenvironmentThe biological principlesexploited . include . The use of a compoundoptic design generatinga panoramicview. The , layout is nonunifonn, with the visual spacein the direction of motion more than elsewhere . denselysampled . Visuomotor control is conductedusing optic flow inducedby the robot' s motion.

Chapter2
. . . . . ' ; : ; ' " " " ' C : : ; ; ' , . ; ; ; ; , .

2.11 Figure Robotic with from , Pichon , and compound . (Reprinted permission Franceshini eye Bianes .) 1992
. Locomotionconsistingof a succession translational of movements followed ' . , by abruptrotations typical of the fly s free-flight behavior . Motion detection circuitry basedon electrophysiologicalanalysis of the . , , ) housefly(FranceshiniRiehle andLe Nestour1989 usinganalogdesign . The use of spacepreservingtopographic(retinotopic mappingsonto the ) . control system . Modeling from an invertebrate as , perspectiveusingan exoskeleton opposed to a backbone . This visual systemwasrealizedin specialized hardware (figure 2.11). This . small compoundeye proved up to the task of supportinglimited real-time navigationin a randomobstaclefield.
2. 5. 3 Cockroach Locomotion are es , Long after the bomb falls andyou andyour gooddeeds gone cockroach will still like . be here prowling the streets armoredcars , - TamaJanowitz

Animal Behavior

Intrinsic Synaptic Currents c- . ~ '/ JC T

Threshold Voltage Firing Frequency

CellMembrane

Figure 2.12 Neuralmodelfrom the Artificial InsectProject This modelusesthe intrinsic currentsto . ' . capturethe neurons dynamicaspects pennitting time andvoltagevariation The cell by membrane usesa ResistorCapacitor(RC) circuit that cantemporallysumthe synaptic andintrinsic inputs. A linear thresholdfunction generates firing frequency the .

conductedat CaseWesternReserve Interdisciplinaryresearch University has studiedthe mechanisms locomotor behaviorin the American cockroach of . In their Artificial Insect Project Beer Chiel, and Sterling ( Beer 1990 Beer , , ; , Chiel, and Sterling 1990 developed neural model more faithful to biology a ) than most usedin neural network research (figure 2.12) . The model includes cell membraneproperties usessynaptic currents and generates , , outputs in ' terms of the neurons firing frequency The individual leg neural control circuitry . is composedof a small collection of theseneuronsbasedupon a biologically derived model for walking ( pearson1976 . In simulation studies ) , Beer and his colleaguescreatedwithin the artificial insect the spontaneous , generationof gaits (metachronalwaves tripod gaits observedin the natural ) insect The simulation model was endowedwith higher level behavioral . controllersthat usedantennae mouth sensors extractinformation from and to the environment The behaviorsincluded wandering edgefollowing, appetitive . , orientationand attractionto food, and a fixed-action patternrepresenting food consumption(figure 2.13). Thesecontrollers were also modeledat the neurallevel. The overall insectwas capableof exhibiting motivatedbehavior (exhibitedthroughthe buildup of arousaland satiationshownin feeding and ) " a variety of statically stablegaits all " strikingly reminiscent of the natural , animalcounterpart .

2 Chapter [~ ~~~~~~~]
BEHAVIORS

Mouth- Tactile -----. : Mouth Chemical Antenna Chemical

Antenna Tactile

Figure2.13 circles showinhibition . Lineswith darkened behavior schematic cockroach of Simplified . between behaviors a ( ) EventuallyQuinnandEspenschied1993 implemented portionof the neural in simulationmodel on a hexapodrobot about centimeters length with of one a massof approximately kilogram (figure 2.14 A ). The aspects the biological ' . model concerningleg control were alsoimplementedThe robot s performance . in confirmedthe locomotiongaitsobserved the simulationstudies et ) biologically inspiredhexapod Espenscheid al. ( 1994 later createda second 5 robot with a massof approximately kilogramsandabout centimeters long (figure 2.14 B), capableof a continuousrange of insect like gaits and Whereas earlier robot was capableonly of the of navigatingirregular terrain. to straight line motion, this newerversionwasgeneralized handleboth lateral . androtationalmovements

Animal Behavior

Figure 2.14 es: Roboticcockroach (A ) showsthe first version which waslater refinedinto the robot , . shownin ( B) . ( photographs , courtesyof R. Quinn, R. Beer H. Chiel, andR. Ritzmann)

60

2 Chapter

2.5.4 Primate Brachiation Actionis atbottom swinging flailingof thearms regalJ one' s balance keep a and to I and afloat . - EricHoffer Another interestingaspectof animal behaviorthat influencedthe designof robotic systemsinvolves a mobile robot that travels by a rather unconventional means Most mobile vehicleseither have legs wheels or tracks but . , , , researchers NagoyaUniversity in Japanhaveconstructed mobile system at a that swings from limb to limb (brachiatesin the style of a long-armedprimate ) such as a gibbon (figure 2.15 A ). The researchers designeda heuristic controller that enablesthe two-link brachiatingrobot (figure 2.15 B) to learn motion sequences trial-and error methodsIt can, afterlearning . appropriate , by success fully catcha targetbar from any initial stateand continuelocomotion bars catch usingthe alonga seriesof spaced . It canalsorecoverfrom a missed , initial statestrategyto beginagain . The level of influenceof the underlying animal behavioralstudiesfor this research markedlydifferentthanfor the cockroach is modelwe havejust stud Led In the insectwork an attemptwas madeto model closely the underlying . , neuralcontrol algorithmsresponsible the animal' s locomotorbehavior In for . the brachiationwork, we seethat no effort is madeto be faithful to the neuro tead physiologyof the primatethat hasmotivatedit. In~ , only the mostoutward of , aspects locomotor behaviorare involved with the ariimal studiesserving solely asinspirationfor the creationof this type of robot.

2.5.5 RoboticHoneybee
That which is not goodfor the bee hive cannotbe goodfor the bee . - MarcusAurelius Research involving communication via dance in the honeybee (Kirchner and Towne 1994) provides an interesting twist on the relationship between robotic and animal behavior. Honeybees have long been thought to convey infonna tion regarding the whereabouts of food source discoveries in their environs by a waggle dance in the hive. The question was whether bees used sound in addition to their dance to convey location infonnation . The debate was resolved after the consttuction of a robotic ~oneybee capable of both singing and danc-

Animal Behavior

(A)

Figure2.15 from with robot brachiationB Brachiating . (Figures .( ) (A) Gibbon reprinted permission ffiEE.) . SaitoFukudaandAraj 1994e 1994 , ,

Chapter
' . ing in a mannersimilar to that of a live bee The bees body was madefrom brassand coveredwith beeswax The wings were constructed . from piecesof razor bladescapableof vibrating via an electromagnetA long rod attached . the body to motorscapable producingthe waggledanceautomaticallywhen of connected computers to . The resultsindicatedthat the mechanicaldancingbee was capableof recruiting beesto fly in the particulardirection that the robot' s danceindicated . This occurredonly when the robot' s wings were vibrating, indicating that soundplayeda role in the communicationIntroducingvariationsin the robot' s . danceand observingthe effect upon the foraging beesprovidedadditionalinformation aboutthe natureof the communication contained the danceitself. in This researchexampleillustratesthat the relationshipbetweenrobotics and the study of animal behavioris mutually beneficialrather than one sided as , roboticscancontributeto the studyof animalbehaviorin additionto benefiting from it.

SUMMARV 2.6 CHAFfER

Animal behavior provides a definition for intelligence . an existence proof for the creation of intelligent mobile systems. models that roboticists can mimic or from which they can draw inspiration . Neuroscience provides a basis for understanding and modeling the underlying circuitry of biological behavior. . The roboticist can view neuroscience from many different levels: . at the cellular level of neurons . at the organizational level of brain structure . at the abstract level based on computational models (e.g ., schemasand neural networks ) derived from the above. . Psychological models focus on the concept of mind and behavior rather than the brain itself . . Various (often opposing) psychological schools of thought have inspired roboticists : . Behaviorism : using stimulus response mechanisms for the expression of behaviors an agent has with its envi ' . Cognitive psychology : using computational models to describe an agent s behavior within the world . . . . .

AnimalBehavior . Ethology is concernedwith the behavior of animals within their natural world. . The definition of behavioral classes , including reflexes, taxes and fixedaction patterns provides a useful language for operationalizing robotic , behavior . . Innatereleasing mechanismsreferredto earlierasschemasprovidea means ( ) for coordinatingmultiple competingbehaviors especiallywhen coupledwith , lateralinhibition. . The conceptof an ecologicalniche enables roboticist to considerhow a the robot is positionedwithin its overall environment how it canbe asuccessful and competitorwithin the world. . Many robotic systemshave beenheavily influenced at various levels by , , studies Examplesinclude ant chemotaxis fly vision, cockroach . , biological locomotion primatebrachiation androbotic honeybees . , ,

Chapter Robot

3 Behavior

The greatend of life is not knowledge but action . , .- ThomasHenry Huxley We really only know whenwe don' t know; with knowledge doubtincreases , . , - Johann von Goethe Wolfgang For in muchwisdomis muchgrief, andhethat increaseth knowledge - Ecclesiastes 18 1: sorrow .

Chapter are Objectives 1 understandto encode .Towhat that express . be 2 learnbehaviorsand .To robotic used these the can methods behaviors coordinating . for and multiple .Tomethods choices 3 learncomposingto behaviors . 4 robotic ofdesignbehavior .To a understanding obtain the related basic based . systems
3.1 WHATAREROBOTIC ? BEHAVIORS
' After developingan understanding behaviors biological basisin chapter2, of we now studyhow to express concepts the and formalismsof behaviorbased robotic systemsIt is importantto remember biological studiesarenot nec . that for serveasinspirations , essarilyviewedasconstraining robots but nonetheless for design .

Chapter3

is the way to view simplerobotic behaviors by adoptingthe Perhaps easiest . advocated thebehavioristschoolof psychologyA behavior simply , by concept us how . put, is a reactionto a stimulus This pragmaticview enables to express . a robot should interact with its environment By so doing, we are confining . in ourselves this chapterto the studyof purely reactiverobotic systems
3.1.1 Reactive Systems we If we had more time for discussion would probablyhavemadea greatmany more
mi ~take~

- Leon Trotsky

reactive

robotic abstract

system

tightly

couples or time

perception history .

to

action

without

the

use

of

intervening

representations

Reactive robotic systems have the following characteristics: . Behaviors serve as the basic building blocks for robotic actions. A behavior in these systems typically consists of a simple sensorimotor pair , with the sensory activity providing the necessaryinfonnation to satisfy the applicability of a particular low - level motor reflex response. . Use of explicit abstract representational knowledge is avoided in the generation . of a response Purely reactive systems react directly to the world as it is sensed avoiding the need for intervening abstract representational knowledge , . In essence what you see is what you get. This is of particular value in , highly dynamic and hazardous worlds , where unpredictability and potential hostility are inherent. Constructing abstract world models is a time consuming ' and error -prone process and thus reduces the potential correctness of a robot s action in all but the most predictable worlds . . Animal models of behavior often serve as a basis for these systems We have . seen in chapter 2 that biology has provided an existence proof that many of the tasks we would like our robots to undertake are indeed doable. Additionally , the biological sciences, such as neuroscience, ethology , and psychology , have elucidated various mechanisms and models that may be useful inoperational izing our robots. . These systems are inherently modular from a software design perspective. ' This enables a reactive robotic system designer to expand his robot s compe-

Robot Behavior

without redesigning discardingthe old. This or tencyby addingnewbehaviors accretionof capabilitiesover time and resultantreusability is very useful for . constructing increasinglymorecomplexrobotic systems are Purely reactivesystems at oneextremeof the robotic systems spectrum , (section 1.3). In subsequent chapters we will see that it may be useful to add additionalcapabilitiesto reactivesystemsbut for now we focus on these , . simplersystems

3.1.2 A Navigational Example

to . Let us constructan examplewith which we canframe the discussion come to . Considera studentgoing from one classroom another A seeminglysimple ' . . task, at leastfor a human Let s examineit more closely and seethe kinds of thingsthat areactuallyinvolved. Theseinclude 1. gettingto your destination from your currentlocation 2. not bumpinginto anythingalongthe way 3. skillfully negotiatingyour way around other studentswho may have the sameor different intentions of 4. observingcultural idiosyncrasiese.g., deferringto someone higher priority ( if in conflict with priority determinedby age or gender in the United , States on passing the right, etc.) elseis necessary 5. copingwith changeanddoing whatever So what soundssimple (getting from point A to point B) can actually be quite complex(figure 3.1), especiallyin a situationwherethe environmentis . not controllableor well predicted Behaviorbased roboticsgrewout of the recognitionthat planning no matter , Burns: The best how well intentioned is often a wasteof time. Paraphrasing , . . laid plans of mice and men oft go astray Oft is the keyword here Behavior basedrobotic systems provide a meansfor a robot to navigatein an uncertain and unpredictableworld without planning by endowingthe robot with behaviors , and that deal with specificgoals independently coordinatingthem in a purposefulway.

3.1.3 Basisfor RoboticBehavior

Where do robotic behaviorscome from? This primary question leads to a to seriesof subsidiaryquestionsthat must be answered provide a robot with behavioralcontrol:

3 Chapter YHI PAR SIDI LARSON ByGARY

ftgure 3.1 . 9 Thingsmay be harderthanthey seem ( TheFar Side< 1993Farworks Inc. Distributed , . . .) by UniversalPress Syndicate Reprintedwith permission All rights reserved

RobotBehavior . . . . What are the right behavioral building blocks robotic: systems for ? What really is a primitive behavior? How are these behaviors effectively coordinated? How are these behaviors grounded to sensors and actuators?

Unfortunately there are currently no universally agreed-upon answers to these questions. A variety of approaches for behavioral choice and design have arisen. The ultimate judge is the appropriateness of the robotic response to a given task and environment. Some methods currently used for specifying and designing robotic behaviors are described below. 1. Ethologically guided/ constrained design. As previously mentioned , studies of animal behavior can provide powerful insights into the ways in which behaviors can be constructed. Roboticists can put models generated by biological scientists to good use. One such example comes from Arbib and House' s ( 1987) studies of the navigational behavior of the toad and its relationship to Arkin ' s ( 1989a) schema based robotic navigational system (section 4.4 ) . In this instance, motion divergence fields are specified for a toad navigating amid a collection of poles toward a can of worms . This model provides an analogous means for representing robot behaviors using a modified potential (force ) field method (figure 3.2) . The key phrase for the design process here is " etho" logically guided : consulting the biological literature for classifications , decompositions , and specifications of behaviors that would be useful for robotic but not necessarily being constrained by them. systems, Other researchers epitomized by Beer ( Beer Chiel , and Sterling 1990) (discussed , , earlier in section 2.5.3), look toward high -fidelity models of the neurological substrateof an animal ( in Beer ' s case, the cockroach) in their attempt to emulate an appropriate behavioral responseby a robot . These scientists choose to deliberately constrain their behavioral models to match those of the animal under study. In many ways this overconstrains the problem of producing intelligent behavior in a robot , but as a side effect this research can potentially answer interesting questions regarding actual biological behavior, for example in terms of predictive modeling . The methodology for designing ethologically guided/ constrained behaviors is illustrated schematically in figure 3.3. A model is provided from a scientific study, preferably with an active biological researcher in tow. The animal model is then modified as necessary to realize it computationally , and is then ' grounded within the robot s sensorimotor capabilities . The results from the robotic experiments are then compared to the results from the original biological studies, and either the biological model or its robotic alter ego are

3 Chapter
-

""
- . ~

./ Worms " t

- . -

.,..-

, .
...... ~

t .
..

.
..

..............-.-

t t , t ....... ~ . . . . . . -. . - ............. Obstacle Fence ...... . . + + ~ . ~

.......

" .... ""

t
~

)"

8 ------- 8 Toad
(A) Figure3.2

Robot Behavior

) Figure 3.2 (continued ' a Toad /robot navigationalmodel (A ) Represents model of a toad s attractionto a can animalpotentialfor motion. from a pole fence and an egocentric of worms avoidance , , the The vectorsrepresent most likely direction of motion for the animal at eachpoint animal with experimentallyobserved . in space This model was shownto be consistent for a data(after Arbib andHouse1987 . (B) Represents setof robotic behaviors similar ) and : circumstancesavoid static obstacles move to- goal. An egocentricpotentialis not . in needed this similar, yet different representation ,

Chapter3
~

OmS 000 ~

Figure3.3 for Design guide! methodology ethologically < I systems(dotted lines indicate optional ). pathway modifiedor enhanced an attemptto produceresultsmorein agreement in with the original animaldata The resultsof theseexperiments . havetwo customers : roboticists who can use theseinsights to produceevenmore intelligent machines , , andexperimental , biologists who canusethemto developandtesttheir theoriesof animalbehavior . ' 2. Situated . that activity- based design Situated activity means a robot s actions arepredicated uponthe situationsin which it finds itself. Hencethe perception problem is reducedto recognizingwhat situation s) the robot is in and then ( . choosingone action (out of perhaps ) many to undertakeAs soonas the robot finds itself in a new situation it selectsa new and more appropriateaction. , Thesemanifold situationscanbe viewedas microbehaviorsthat is, behaviors , . specifiedand useful only in very limited circumstancesDesigning a robot basedon this methodologyrequiresa solid understanding the relationship of betweenthe robotic agentand its environmentThe designstrategytypifying . this methodappears figure 3.4. in

GO

Wc

. ta

cO

Robot Behavior

. Assess Agent Environmen Damics Partition into Situations Create Situational Responses Import Behaviors to Robot Run Robotic Experimen Evaluate Results

Enhance , , Expand Correct Behavioral Responses

8 Figure3.4 . Situated activitY designmethodology

Arbitrarily complex situations can be created and specified that may have no biological basis. Pengi (Agre and Chapman 1987) is a system that char. acterizes situations by their indexical -functional aspects Indexical refers to ' refers to a robotic agent s what makes the circumstances unique , functional intended outcome or purpose in a given situation . This system uses lengthy . phrasesto characterize particular situations that demand certain responses For the situations from the original Pengi somewhat), theexample ( paraphrasing block - I - need- to -kick -at- the- enemy-is -behind -me is a situation that requires the ' - - agent to backtrack to obtain the object in question. I ve run into the edge of -wall requires that the robot turn and move along the edge. These situations the can be highly artificial and arbitrarily large in number. Coordination in Pengi

Chapter3 , is handled by an arbittation mechanism, where one of the candidate actions is chosen (there may be many applicable) and executed. Indeed, one candidate action may be in conflict with another. Hopefully the best action is chosen, but there is no guarantee, as no planning is conducted, nor does the mechanism project the consequencesof undertaking any action. Assuming that there is no limit to the number of situational conditions that can be enumerated leads us to a more expansive version of this theory of situated activity : universal plans. Universal plans, as developed by Schoppers 1987), require the robotic agent to have the ability of recognizing each unique ( situation for what it is and then selecting an appropriate action for each possible world state. These universal plans cover the entire domain of interaction , use sensing to conduct the classification , and presuppose no ordering on the situations or even the type of situations that might arise for that matter ( Schoppers 1989) . Sensing is conducted continuously , so situational assessment and , thus appropriate response, is continuously reevaluated. To deal with the issue of the sheer bulk regarding the vast number of possible situations , the idea of caching plans is forwarded . Despite this technique, universal plans have encountered significant criticism (e.g ., Ginsberg 1989) predominantly due to the immensity of the numbers of plans required and the potential irrelevancy of most. Even the harshest critics acknowledge that more limited versions of the situated activity paradigm have utility , even when it is designed to include not only routine situations but a wide range of contingent ones. The argument, however, that an enumeration of every possible situation (i .e., universal plans) is impractical at best and mathematically inttactable at worst is a valid one. Reactive action packages (RAPs ) (Firby 1989) constitute an unusual variant on situation -driven execution. As with the other methods, the current situation provides an index into a set of actions regarding how to act in that environment . RAPs , however, operate at a coarser granularity than the other situated-action approaches and provide multiple methods of acting within a given context. RAPs consist of a set of methods specific to a task-situation , and for each of those methods, a sequenceof steps to accomplish the task is provided (a kind of " sketchy" plan ) . RAPs differ from most reactive systems, however, in that they are not truly behavior based ( but rather task based) and in that the system relies heavily on a strong explicit internal world model. 3. Experimentally driven design. Experimentally driven behaviors are invariably created in a bottom - up manner. The basic operating premise is to endow a robot with a limited set of capabilities (or competences) , run experiments in the real world , see what works and what does not , debug imperfect behaviors, and

RobotBehavior

Build Minimal System -'f Exercise Robot -.t Evaluate Results ~ Add New Behavioral Competence

8
Figure 3.5 for . Experimentallydriven methodology behavioraldesign

thenaddnewbehaviors iterativelyuntil the overall systemexhibitssatisfactory 3.5). (figure performance in An excellentexampleof this designparadigmappears Brooks' ( 1989a ) -basedcontroller for a leggedwalking robot. work on the designof a behavior Initially the robot (panel(A ) in figure 3.6) was provided with the ability to for standup and conducta simple walk. This worked adequately smoothterrain . to but posedproblemsasthe robot attempted walk over irregularsurfaces the requirements this extendedcapability force balancingwas of Basedon , addedto modify the leg controllersand help the robot maintaina steadyposture in . Whiskers(protrudingsensors the front of the robot) were then added to provide more warningto the control systemto deal with largerobjectsthat in . therobot needed climb over A final problemwasnotedinvolving balance to situationswherethe robot washeavily tilted fore or aft (pitching . To compensate ) , an inclinometercoupledwith new pitch stabilizationcode was addedto as over highly irregular provide evenbetterperformance the robot maneuvered terrain. It wasthendecidedto allow the robot to track warm objectssuchas wereadded coupledwith a newbehaviorto provide , , people soinfraredsensors was , incrementallybased uponthe competencies added prowling. Eachof these and resultsof previousexperiments the goal of providing greaterutility for the of . . robotic system Section4.3 detailsthe development this system introducedinto reactiverobotic systems In Paytonet ale1992 fault toleranceis , through the design of suitablebehaviorsthat can handleunanticipated

Chapter3

(A)

Figure3.6 . of Brooks (B) Genghis - a .) 1I (A) OriginalGenghis( photograph courtesy Rodney robotichexapod successor theoriginalGenghis to . (photograph , commercial courtesy . of IS Robotics , Somerville ) , MA.

RobotBehavior

" as . , contingencies they arise This designmethodology dubbed Do whatever " hasa works, a goal of generating sufficiently generalsetof low-level behaviors ' that when activated copewith eventsbeyondthe initial designers vision can . Redundancy the ~ey feature that is, allowingthingsto be accomplished is , in more than one way andthen designinga controllercapableof selectingthe mostsuccessful behaviorfor thecurrentsituation In a sensethis featureis also . , embodied theRAPssystem in described , previously in which multiple methods areusedto accomplish task a . Ferrell ( 1994 developeda complex control systemfor anotherwalking ) hexapod potentiallysuitableasa lunaror Marsroverusingthis bottom up strategy . In this implementation 1,500 concurrent es locomotion , process supported over rough terrain and providedthe requisitesensingwith a significantlevel . without any relianceon , of fault tolerance The entire systemwas constructed simulationtechnology Earlier, Connell ( 1989a demonstrated efficacy of . the ) this experimental methodwith the designof a mobile manipulator(figure 3.7). In particular the arm controller for this systemconsistedof fifteen independent , behaviors , , capableof finding a sodacan, then grabbing transporting and it at anotherlocation . depositing Whatever designbasis a genericclassification robot behaviors be the of can , usedto categorize differentwaysin which a robotic agentcaninteractwith the its world: Exploration / directional heading based behaviors (move in a general direction )

wandering Goal- oriented appetitive behaviors (move towards an attractor ) discrete object attractor area attractor Aversive / protective behaviors (prevent collisions ) avoid stationary objects elude moving objects ( dodge, escape ) aggression Path following behaviors (move on a designated path ) road following hallway navigation stripe following Postural behaviors balance stability

3 Chapter

~~-

3.7 Figure

Herbert- a mobile manipulator .

courtesyof JonConnell.)

Social

/ cooperative

behaviors

sharing foraging

~~ / hA king Tding
behaviors modification

Teleautonomous

( coordinate

with human operator )

influence
behavioral

Perceptual saccades
visual ocular

behaviors

search reflex es

Robot Behavior Walking behaviors ( for legged robots ) gait control

behaviors ( for arm control ) Manipulator specific reaching Gripper / dextrous hand behaviors ( for object acquisition grasping enveloping

Perceptual support is required to implement any of these behaviors. Chapter 7 describes how perception can be tailored to behavioral . need

3.2 EXPRESSIONBEHAVIORS OF
Severalmethodsare availablefor expressing robotic behavior This book employs . -response SR) diagrams functional notation and finite three Stimulus : , , ( . will be usedthroughoutthe text stateacceptor FSA diagramsThesemethods ( ) in representing variousbehaviorbasedsystemsSR diagramswill be usedfor . of specificbehavioralconfigurationsfunctional notation , graphicrepresentations for clarity in designof the systemsandFSAswhenever , temporalsequencing is . of behaviors required

.1 3.2 StimulusResponse Diagrams

Stimulus response SR) diagramsare the most intuitive and the least formal ( methodof expressionAny behaviorcanberepresented a generated . as response . to a givenstimuluscomputed a specificbehavior Figure3.8 showsa simple by . SR diagram an Figure 3.9 presents appropriateSR diagramfor our navigationalexample (section3.1.2). Here five different behaviorsare employedin the task of into a coordination . getting to the classroomThe outputsof eachbehavioris channeled here an overall mechanismschematized ) that produces appropriate ( motor response the robot at anypoint in time giventhe currentexistingenfor of es vironmentalstimuli. Section3.4 discuss further the problemandmethods . coordinatingbehaviors

Stimulus

Response

Chapter3

3.9 Fi2Oft
SR diagramfor classroom navigationrobot. 3.2.2 Functional Notation Mathematical methods can be used to describe the same relationships using a functional notation , b (s ) = r , meaning behavior b when given stimuluss yields response r . In a purely reactive system, time is not an argument of b , as the behavioral response is instantaneous and independent of the system' s time history . A functional expression of the behaviors necessaryto carry out our example navigational task of getting to the classroom would appear as: coordinate - behaviors [ move- to - classroom ( detect - classroom - location avoid - objects ( detect - objects ) , dodge students (detect - students ) , - to - stay right on path ( detect - path ) , - to - elders (detect - elders ) defer ] = motor - response ),

The = motor - response is usually implicit and is generally not written when using this notation.

"

detected elder

_.

_ .

detected path

~.

detected student

detected object

class location

J .

_ .

_ .

. .

_ .

Action

RobotBehavior Each of the five behaviors listed can produce an output depending on the current environmental stimuli . A coordination function determines what to do with those outputs (e.g ., selecting one of them or combining them in some meaningful way ) . It is not a trivial problem to ensure that the outputs of each ' behavioral function are in a form that can be coordinated , as section 3.4 will detail . Coordinated functions can also be the arguments for other coordinating functions . For example coordinate - behaviors [ coordinate - behaviors (behavioral - set - l ) coordinate - behaviors ( behavioral - set - 2 ) coordinate - behaviors ( behavioral - set - 3) ] where each of the behavioral setsis a set of primitive behaviors. Clearly this notation readily permits a recursive formulation of behavior, ultimately grounded in physical robotic hardware but able to move upward into arbitrary levels of abstraction. Functional notation has an interesting side effect in that it is fairly straightforward to convert this representation into a computer program . Often a functional programming language such as LISP is used, although the C language also enjoys widespread usage.

3.2.3 Finite StateAcceptor Diagrams

Finite stateacceptorsArbib, Kfoury, andMoll 1981 havevery usefulproperties ( ) when describingaggregations sequences behaviors(section3.4.4). and of They makeexplicit the behaviorsactive at any given time and the transitions betweenthem. They are lessuseful for encodinga singlebehavior which results , in a trivial FSA (figure 3.10). In figure 3.10, the circle b denotes statewherebehaviorb is activeand the . is accepting stimulusinput. The symbola denotes input in this case all any M can be specifiedby a quadruple (Q, <, qo, F ) 5 A finite state acceptor with Q representing setof allowablebehavioralstates < being a transition the ; 5 function mappingthe input and the currentstateto another or eventhe same , , a state qOdenotingthe startingbehavioralconfiguration and F representing ; ; setof accepting statesa subset Q, indicatingcompletionof the sensorimotor of , in task. < can be represented a tabular form where the arcs representstate 5 transitionsin the FSA andareinvokedby arriving stimuli.

Chapter3

.5 q input ,input .5 ) (q ba b

a transitionfrom one stateto another and the resulting state For this trivial FSA, all . , . inputsresult in the samestate

For the trivial exampleshownin figure 3.10: M = {{b}, ~, b, {b}} .

FSAsarebestusedto specifycomplexbehavioral control systems whereentire setsof primitive behaviorsare swapped andout of executionduring the accomplishment in of somehigh-level goal (Arkin andMacKenzie1994 . Gat and ) Dorais ( 1994 havealso expressed needfor sequencing the behaviors FSAs . ) a readymechanism express to theserelationships betweenvariousbehavioral provide sets and have been widely used within robotics to expresscontrol . , systems In their MELDOG system Tachi and Komoriya ( 1985 use an automaton ) actionsthat shouldbeexecuted variousplaceswithin at mapto capture theworld. Similar examples existof FSAusagefor guidingvision-based robots and Kabuka 1991 Tsai and Chen 1986 . Brooks ( 1986 has also used ; (Fok ) ) a variation augmented finite statemachines(AF SMs), to expressbehaviors , within his subsumption architecture section4.3). In this text, however we use ( , thenotationdeveloped Arkin andMacKenzie1994based the formalisms in on in described Arbib, Kfoury, andMoll 1981 . The classroomnavigationexamplecan also be expressed with FSAs, although the result is of a decidedlydifferent character(figure 3.11 . This example ) has four different states start journey, lost, and at-class The last two : . ,

Robot Behavior other

-at not -class

all

Start -class reached At CIass all

3.11 Figure
. FSA representing classroom navigationexample

areterminal states lost is abnormal at-classnormal. Journeythe main behavioral : , , state actuallyconsistsof an assemblage coordinated collection of the , (a ) five other low-level behaviorsmentionedearlier (move to-classroom avoid , -students stay to-right-on-path, and defer to-elders . Specifically , , ) objects dodge , M = {{start, journey, lost, at-class} , 8, start, { lost, at-class} } . The FSA provides us with a higher level of abstractionby which we can betweensetsof behaviors . the express relationships

3 Chapter
To further illustrate this, let' s look at an evenmore complexexample Figure . 3.12 depictsan FSA constructed a robot usedin a competitionconducted for . , by the AmericanAssociationfor Artificial Intelligence Here a collection of high-level behaviorseachrepresented asa state encodes , , schematically the robot' s goal of moVingaboutan arenalooking for ten distinct poles then , . moving to eachof thosepolesin sequenceThis robot hasthreemajor behavioral states wander move to- pole, and return to-start Move-to-pole consists . , , of a subsetof actionsfor selectinga pole, orienting the robot so it points toward the pole, movingto the pole, andtrackingthe pole visually during motion until it is reachedIn this case . , M = {{start, lind -nextpole, moveto-pole, wander , -to Return -start, halt } , 8, start, { halt } }, where8 containsthe transitioninformation depictedin figure 3.12. The FSA showsthe sequencing betweenbehaviorsas the robot carriesout its mission 3.13). More details on this task and robot can be found in Arkin et (figure al. 1993 Incidentally the use of finite statedescriptionsin the University of . , Southern California' s PhonyPonyprojecton quadruped locomotion(McGhee 1967 is probablythe first exampleof their applicationfor specifyinga robot ) control system .

3.2.4 FonnalMethods
Fonnalmodelsfor behaviorbased roboticscanpotentiallyprovidea setof very usefulpropertiesto the robot programmer : . They canbe usedto verify designer intentions . . They canfacilitate the automaticgeneration robotic control systems of . . They provide a complete common languagefor the expressionof robot behavior . . They provide a frameworkfor conductingformal analysisof a specificpro' / . , , gram s properties adequacyand or completeness . They provide supportfor high-level programminglanguage . design Severalformal methodshavebeendeveloped specifyingand designing for behaviorbased robotic systemsA brief reviewof two representative . strategies is presented below .

RobotBehavior other

Find next pole

Return to-start

ISext lSol inpu ,p (- ex q qinput -) find n start star -start -oole find n ole p -o return t move p t wande halt
-up all
wander

Halt

compete other all-poles found no- pole-found pole-selected not-at-start at-start lost at-pole not-at-pole time-not-up timeout all

return to- start wander move to- pole return to- start halt find-next-pole find-next-pole move to- pole wander find-next pole halt

3.12 Figure robot . FSA representing competition example

Chapter3

3.13 Fiaure Robotexecutingbehaviorsat competitionarena .

3.2.4.1 RS
the ) Lyons and Arbib ( 1989 developed RS (robot schema model as a method ) for expressing distributedsensordriven robot control programs Aprocessal . is usedthat permits the compositionof a network of process called es gebra schemas (behaviors Processcompositionoperatorshave been defined as a ). basisfor creatingthesenetworks which include methodsfor conditional sequential , , . , parallel, and iterative structures Preconditionsare establishedfor coordinationoperators ensurea smoothflow of control during execution to . In particular a port automata model hasbeenadoptedas the underpinning , for expressing relationships the between schemasPortautomata be viewed . can as an extension FSAs with supplemental of formal methodsfor specifyingthe interconnections betweenstates Schemas . communicatewith each other via -outputports using synchronous . predefined input message passingtechniques ' s behavioral A schema its to , description which encodes response any input are via , messagesfully determinesits action. Schemas aggregated a nesting mechanismtermed an assemblageEach assemblage . recursively encodesa networkof schemas other assemblages or . unique

Robot Behavior

A high-level algebraic RS encoding for the navigational example used throughoutthis chapterwould be: Class- going - robot = ( Start - up ; (done? , Journey) : At - classroom) Journey = (move to - classroom , avoid - objects , dodge students , - to - right - on- path , defer - to - elders ) stay , Translating the Class- going - robot consistsof a robot that, beginning from an initial start up state sequentially transitionsto the Journey state(the , sequential operatoris denoted;) and which then remainsin the Journey state with a concurrentmonitor processcheckingfor arrival at the classroom(the concurrencyoperatoris denotedwith a ,). If the robot is at the classroomit transitionsto the At - classroom state(denotedby the conditionaloperator:). of executionof the behaviorsspecified Journey consists the concurrent during travel from onelocationto the next. This methodcombines of advantages both the functional andFSA methods into a singlesyntax . Thereis far moreto theRSmodelthanwhatis presented . The interested here readeris referredto Lyons and Arbib 1989 Lyons and Hendriks 1994 and , , . Lyons 1992for moredetails
Automata 3.2.4.2 Situated

The situated automata model developed KaelblingandRosenschein1991 , ( ), by the recognizes fundamental relationshipan agenthasasa participantwithin its environmentThe model employslogical formalismsasunderpinnings the . for of circuitry that corresponds a robot' s goalsand intentions The use to . design of logic enablesreasoningover the system which can lead to the establishment , of provableproperties(Rosenschein Kaelbling 1987 an important and ), . ), goal for the designerof any type of system Rex (Kaelbling 1986 a LISPbasedsystem was the first languageto embody the basic tools to generate , for specifications synchronous digital circuitry embodyinga reactivecontrol . program Gappsis a more recentlydeveloped languagethat enablesgoals to be specifiedmore directly and is inherently easierto use akin to a higher , in conventional . Goalsare either achieved , generation language programming executedor maintained Achieved goals are thosethat should be eventually : realized executedgoals are thosethat should be done now; and maintained ; . , goals are thosethat shouldbe preservedas they have alreadybeenattained aredefinedin a LISP-like format that correspond thesethreegoal to Operators : states ( ach goal ) for achieve (do goal ) for execute and (maint goal ) for , ,

Chapter3

maintain The logical booleanoperators , or , not , if are usedto create . and -level goals . higher Circuits aregenerated from thesehigh-level goal expressionsStandard . logical methods be us.ed compilethe high-level circuitry into a collectionof can to . digital logic gates This type of formalismprovidesa very concrete grounding onto actualdigital hardware creatingsituatedautomata for robots . A simplified Gappsspecificationfor our ongoingclassroom navigationexample would be: (defgoalr (ach in - classroom) ( if (not start - up) (maint ( and (maint (maint (maint (maint (maint ) ) ) )

move to - classroom) avoid - objects ) dodge students ) - to - right - on- path ) stay defer - to - elders )

This encodingstatesthat the robot is to achievethe goal of being in the classroom If the robot is not in start - up state then it is to journey to the . , location by maintaining the concurrentgoals of moving to the classroom , . , , avoidingobjects dodgingstudentsstayingto theright, anddeferringto elders Methodsalso exist to prioritize goals within the Gappslanguageshould the needarise As in the casewith RS there is far more to the Gappslanguage . , thancanbe discussed , sothe interested here readeris referredto Kaelblingand Rosenschein 1991for additionalinformation. underwater vehicle (UUV ), described Gappscircuitry wasusedfor an unmanned in Bonasso1992 The basicgoalsestablished this systemwere . for . (maint not - crashed) which established subgoalof ( ach avoid nearest the obstacle ) alongwith otherbehaviors suchasavoidingcollision with the oceanbottom. . ( ach wander endowed robot with explorationcapabilities the . ) . (ach joystick goal - point ) alloweduserdirectedinput to control the direction of the robot. The first two of thesebehaviorsprovided the UUV with the ability to navigate safely in a water tank. Additional behavioralgoals were describedus-

Robot Behavior

ing Gapps and tested in simulation , including ( maint best - heading ) and mission - specific tasks such as ( ach record - thermal - vent - event ) and (ach quiescence ) .

3.3 HERA VIORALENCODING

To encode the behavioral response that the stimulus should evoke, we must create a functional mapping from the stimulus plane to the motor plane. We will not concern ourselves at this point as to whether a behavioral response is appropriate to a given stimulus , only how to encode it . An understanding of the dimensionality of a robotic motor response is necessary in order to map the stimulus onto it . It will serve us well to factor the robot ' s motor response into two orthogonal components: strength and orientation . . Strength denotes the magnitude of the response, which mayor may not be related to the strength of a given stimulus . For example, it may manifest itself in terms of speed or force . Indeed the strength may be entirely independent of the strength of the stimulus yet modulated by exogenous factors such as intention (what the robot ' s internal goals are) and habituation ( how often the stimulus has been previously presented) . We will later see that by controlling the strength of the response to a given stimulus , inroads are created for integrating goal -oriented planning into behavioral systems (chapter 6) as well as introducing the opportunity for adaptive learning methods (chapter 8) . . Orientation denotes the direction of action for the response, (e.g ., moving away from an aversive stimulus , moving towards an attractor) . The realization

' of this directionalcomponent theresponse of of requiresknowledge the robot s 's . kinematics It mayor may not be dependent the stimulus strength on .

In,-isofobje kinemat the,.all scienc inkine motio of ,timeinclu general asp 's andexte acceler .objec This includ rob ,inof ,the position velocity parti and bthat . geometric physica Dyna prope includeto themotio ofased the forces study produ

A behaviorcanbeexpressed atripleS , R, .8 whereS denotes domain as the ) of all interpretablestimuli, R denotes rangeof possibleresponsesand .8 the , denotes mapping.8: S - + R. the

Chapter3

R- Range Responses of this r "R , Refining . further the instantaneous response (whererE ) of a -based behavior reactive can as vector system beexpresseda sixdimensional of sixsubcomponent . Each thesubcomponent vectors of vectors consisting encodes magnitudethe the of translational orientational and for responseseach of thesixdegreesfreedom motion ageneral of of of mobile . robot

degree

of , with within

freedom respect the

( DOF ) to .

refers of

to

one

of

the

set

of

independent to specify an

position ' object s

variables position

a frame

reference

, necessary

world

An unconstrained rigid objecthassix OOFs, r = [x , y , Z, 6, 4 , 1 ], > /1 wherethe first threecomponents r represent threetranslationaldegrees of the of freedom (x , y , z in three dimensionalcartesiancoordinates, and the last ) threecomponents encode threerotationaldegrees freedom(6 for roll , 4 the of > for pitch, 1 for yaw). Often pitch is alternativelyreferredto astilt , and yaw as /1 . pan, asin a pan tilt device This is especiallytrue in the contextof controlling the pointing of sensors suchascameras . -basedmobile robots the dimensionalityis often consider For ground , ably lessthansix OOFs. For example a robot that moveson flat groundandcanro,
tate only about its central axis has only three degrees of freedom , r = [x , y , 9 ], representing translation in the cartesian ground plane [x , y ] and the one degree of rotation 8 (yaw, or alternatively pan) .

Another factor that can limit the realization of a generated behavioralresponse is the robot' s non-holonomicity .

A- olonomicmove non hasin,suchabilitie robotthelimited restrictions way ofhorvelocities becau kinematic itasturning constraints on can the robot dynamic ,typically or at . momentwn high
A truly holonomicrobot can be treatedasa massless point capableof moving in anydirectioninstantaneously . Obviouslythis is a very strongassumption anddoesnot hold for any real robot (althoughit is easyto makethis assump

Robot Behavinr

tion in simulation potentiallygenerating resultsregardinga control , misleading 's on an actualrobot). Omnidirectionalrobotsmoving at slow algorithm utility translationalvelocities however (that is, robotsthat can essentiallyturn on a , , dime and headin any direction can often pragmaticallybe considered be to ), holonomic but they are not in the strictestsenseNon-holonomicity is generally . , of greatsignificancewhen thereare steeringangleconstraints suchas in , a car attemptingto park parallel. H the wheelswere capableof turning perpendicular to the curb, parking would be much easier but as they cannot a , , of sequence more complexmotionsis requiredto move the vehicle to its desired location . The constraints can imposedby non-holonomicsystems be dealt with either , during the generationof the responseby including them within the function to , , ,8, or after r hasbeencomputed translatingthe desiredresponse be within the limitations of the robot itself. S- The Stimulus DomainS consistsof the domain of all perceivablestimuli. Each individual stimulus or percepts (whereseS is represented a binary tuple (p , }..) having both a as ) or perceptual classp anda propertyof strength The complete }... particulartype set of all p over the domainS definesall the perceptualentities a robot can . , distinguish that is, thosethings it was designedto perceive This conceptis in as looselyrelatedto affordances discussed section2.3. The stimulusstrength }.. can be definedin a variety of ways: discrete(e.g., binary: absent present or ; : absent weak, medium strong or real valuedandcontinuous . , , ) categorical In other words it is not requiredthat the merepresence a given stimulus of , be sufficientto producean actionby the robot.

Wedefine't' asa thresholdvalue for a givenperceptual classp , abovewhich , a response generated is . Often the sttengthof the input stimulus(J ) will determinewhetheror not to .. and the magnitudeof the responsealthoughother exogenous factors , respond can influencethis (e.g., habituation inhibition, etc.), possiblyby altering the , valueof 't'. In any case if J is nonzero the stimulusspecifiedby p is present , .. , to somedegree .

motor

evoke

to

sufficient

not

but . necessary

is robot based -

stimulus

of behavior a in presence

The response

Chapter3

Certainstimuli may be importantto a behaviorbasedsystemin waysother than provoking a motor response In particular they may have useful side . , effectson the robot, suchas inducing a changein a behavioralconfiguration evenif they do not necessarily induce motion. Stimuli with this property are referred to as perceptual triggers and are specifiedin the samemanneras , , , (p, .. previouslydescribed .1). Here however when p is sufficiently strong the desiredbehavioralsideeffect is produced ratherthanmotion. We returnto our discussion stimuli in the contextof perceptionin chapter7. of 13 The Behavioral Mapping the Finally, for eachindividual activebehaviorwe canfonnally establish map the ping between stimulusdomainandresponse rangethatdefinesa behavioral function 13 where ( 13s) ~ r . 13can be definedarbitrarily, but it must be definedover all relevantp in S. Wherea specificstimulusthreshold 'r, mustbe exceeded beforea response is , for produced aspecifics = (p, A) : * : \ * no response \ 13 (p, A) ~ {for all A < 'r then r = [0, 0, 0, 0, 0, 0] * elser = arbitraryfunction} \ * response \ where [0, 0, 0, 0, 0, 0] indicatesthat no response requiredgiven the current is stimulis . Examples Considerthe exampleof collision avoidance behavior If an obstaclestimulus . is sufficiently far away (henceweak no actual action may be taken despite ), its presenceOncethe stimulusis sufficiently strong(in this casemeasured . by . , evasiveaction will be taken To illustrate intuitively, imagineyou proximity) arewalking on a long sidewalkandyou seesomeone far approaching aheadof , you. In general you would not immediatelyalter your pathto avoida collision, but ratheryou would not reactto the stimulusuntil action is truly warranted . This makessense as the situation may changesignificantly by the time the , you oncomingwalker reaches . She may havemovedout of the way or even turnedby herselfoff the path requiring no actionwhatsoever your part. on , The functionalmappingbetween strength stimulusandthe magnitude the of definesthe design space (strength and direction of robotic motor response ) . for a particular robotic behavior Figure 3.14 depicts two possiblestimulus above Oneis a step . strengthresponse strength plots for the situationdescribed

RobotBehavior

function in which, oncethe distance thresholdis exceededthe action is taken , at maximumstrength The other fonnulation involvesincreasingthe response . strengthlinearly over somerangeof stimulusstrength(measured distance by 's here . Other functionsare of coursepossible The response orientationmay . ) also vary dependingon how the behaviorhas been constructed For example . the motor response be directly awayfrom the detected , may object (strict ), repulsion move away or alternativelytangentialto it (circumnavigation ) go left or right) (figure 3.15 . Associatedwith a particular behavior ,8, may be a scalar gain value g , of r (strengthmultiplier) further modifying the magnitude the overallresponse for a givens: r ' = gr . Thesegain valuesareusedto compose multiple behaviors specifyingtheir by relative to one another(section3.4). In the extremecase g can be , strengths usedto turn off a behaviorby settingit to 0, thusreducingr' to O . The behavioral fall , mappings,8, of stimuli onto responses into threegeneral : categories

. Null: The stimulusproduces motor response no . . Discrete The stimulusproduces response : a from an enumerable of prescribed set choices(all possibleresponses consistof a predefined cardinal set of actionsthat the robot can enact e.g., turn-right, go-straight stop travel-at, , , enumerated speed5). R consistsof a boundedset of stereotypicalresponses for the stimulusdomainS and specifiedby ,8. . Continuous The stimulus domain producesa motor response : that is continuous 's over R range (Specific stimulis are mappedinto an infinite set of . response encodings ,8.) by earlier: For all Obviously it is easy to handle the null caseas discussed s, ,8: s ~ O Although this is trivial , there are instances . ) (perceptual triggers where this response wholly appropriateand useful enablingus to define is , es of perceptual process independent direct motor action. The methodsfor encodingdiscreteand continuousresponses discussed are in turn.

3.3.1 Discrete Encoding

Chapter3
0 0 ~

~ ~ ' -"' 0 w , (/) c 0 Qo . (/) 10 , I}: \tOo .. .c .0) c ' 0 L ~ . .In

0 0 Distance 2 4 from 6 Stimulus 8 ( m ) 10

(A) 3.14 Figure

. pairs. Sensingprovides the index for finding the appropriatesituation The for a situationcan be very simple suchas halt, or more , responses generated a of , complex potentially generating sequence actionsakin to the fixed-action in patternsdescribed chapter2. Another strategyusing discreteencodingsinvolves the use of rule-based . as systemsHere.8is represented a collectionof If -thenrules. Theserulestake the generalform: IF antecedent THEN consequent wherethe antecedent consistsof a list of preconditions that must be satisfied in order for the rule to be applicableand the consequent containsthe motor . to responseThe discretesetof possibleresponses corresponds the setof rules in the system More than one rule may be applicablefor any given situation . . The strategyusedto deal with conflict resolutiontypically selectsone of the potentially many rules to usebasedon someevaluationfunction. Many rule-

Robot Behavior 0 0 ~ """ ~ \.-" 0 ~ ~ U ) c 0 a. o U CO ) ~ ~ II00 ~ .c : +C ) C ~O LoN +U )

0 0 2 Distance 4 from 6 Stimulus 8 ( m) 10

(B) ) Figure3.14(continued
Stimulusdistanceresponse / . strengthplot: (A ) stepfunction, and (B) linear increase

basedbehavioralsystemsencodetheir behaviorsusing fuzzy rules; we will in studythesesystems moredetail in chapter8. section3.2.4.2) usesgoal-reductionrules to encodethe actionsrequired Gapps( to accomplisha task. Here the antecedent specifiesa higher level goal that if necessary will require that certain subgoalsbe achieved Eventually . thesesubgoalstranslateinto specific motor commands or action vectors to , ( usetheir tenninology . One examplefor an underwater robot ( Bonasso 1992 ) ) usedthe following Gappsrule: (defgoalr ( ach wander ) ( if (not (RPVat - wander angle ( ach turn to wander angle ) (ach wander set point ) Here the wanderbehaviorrequiresthat the robot turn to a new wanderangle andthenmoveto a newly established point. set

Chapter3

. . . . . . . . . " . . _ , . . . . . . . . ' . " . " . , . . . . . . . . . . . . . . " . . . . . . . . . . . . rr . " " . . r . ' . . . . . . . . . . . . . . , . . ~ ~ . ~ . . " , " . . . . . . . . " " .. " . . " . " " . " " .. " .. " .. " " " .. .. . . . " . . .

. . . . " " . " " . .

. . .

. . . , " " " . . .

. . . . . . .

. . . I , . . tt " tt

. . . , . , . " " " " tt . . . . .

. . . , . , . " .

. . . . " " " " " ' " .. .

. . . . . . " " " , " .. , , " ~ " " " " , ~ ~ ' " . , . . . . . . . . . . . . . . ~ , " " ~ ~ ~ ~ . .. " " " ' . . " " . ~ " " ' . ~ " . ~ ~ ~ ~ ~ " " . . ~ . . . . . _ " " " " ' . . " . . . . . . . ' . . . . . _ _ _ . . . _ .

. . . . " " . . ' . . . . . . . . . . . . . . ' . . . . . .

" " " , "

" , \ .. \ " \ , " \ , ' , \ tt ,

" " " t " " t fl fl tl '

tt t ,

" .. " .. "

. ' . . .

. "

~ " ~ ~ " ~ -* .. .. " "w -* " ~ . .. , ... ... "w ... ... .. ~ . .. . . . . ... ... . . ... ... . . .. ~~ ~ .. .. . . . . ... ~ ~ ... ... . . .. .. ~ ~ .. . . . . . . .. ... . . ~ .. p ~ ~ ~ ~ ~ ~ ~ ~ " pp , , -... ... ~ .. .. . " P *' - ... . . . . .. . . .. .. . . ... ~ ... . . .. . ~ .. .. .. ... . . ~ .. .. " .. " JJ ~ \ . .. .. .. .. " .. .. Jl ~ ~ ~ .. " " " " ,, " " " J ~ ~ ~ ~ ~ " " " " .. .. .. ~ I -- .. .. " ~ ~ \ .. .. .. .. .. JII " " .. " \ ~ \ ~ ~ .. .. .. .. JI ~ .. " " \ ~ \ ~ .. .. .. .. JIJ ~ ~ ~ ~ ~ .. .. ~ ~ ~ .. ' ~ ~ ~ ~ ~ IIJ ~ ~ ~ ~ " ' I I ' & ~ ~ ~ ~ ~ ~ " " " ' II & & ~ ~ ~ ~ " " " ~ " & " " III . . " " " " , " . . " " " " " , " " . . . , , , , , , , , , I I . . . . . . . I I I , , , , , I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . " " " '" ' ~

(A )

3.15 Figure
In Nilsson 1994 condition action productionrules control the robot. The , the condition (antecedentis basedpartially on sensoryinformation, whereas ) . encodes response the robot to enact The resultingactionsare a for consequent durative insteadof discrete heremeaningthat the actionresulting themselves , . from the invocation of the rule persistsindefinitely (Contrast the durative " of " of / response Move at a speed five meterssecond with the discreteresponse " Move fifteen meters This teleo reactivemethodis somewhat relatedto the ." ) modelsepitomizedby Gapps yet usesdistinguishingformalismsthat , circuitry . canpotentiallyfacilitate analysis for Anotherexampleof discreteencodinginvolvesrulesencoded usewithin architecture Brooks1986 . To facilitatethe useof this particthe subsumption ) (

Robot Behavior

( B) ) Figure 3.15 (continued lWo directionalencodings collision avoidance(A ) repulsive and ( B) circumnavi for : , from the bottom of the figure. , gation with approach

ular approach reactivesystemsthe BehaviorLanguage created Brooks to was , ( -time, rule-basedapproach 1990a. This language embodies real a ) forspecifying and encodingbehavior Using a LISP-like syntax rules are specifiedwith . , the whenever clause : (whenevercondition &restbody1orms ) wherethe condition portion of the rule corresponds to the antecedent and the
& rest to the consequent. Behaviors are consbucted by collecting a set of these real- time rules into a group typically using the defbebavior clause:

Chapter3

es (detbehavior name&key inputsoutputsdeclarations process ) wherethe list of rules appears theprocess location. in es Similar ruleswereusedfor the control of a wheelchairin ConnellandViola . 1990 A loosetranslationof a few of thesebehavioralrules include . Approach IF an object is detected : , beyondspecifiedsonarrange THEN go forward. . Retreat IF an object is nearbyaccordingto sonar THEN movebackward : . , . Stymie: IF all the front infrared sensors indicate an object immediatelyin front, THEN turn left. The entirerobotic systemconsisted fifteen suchbehaviors of encodinga map of direct sensingto motor activity. (Thesebehaviorsstill requirecoordination ping in , which is discussed section3.4) .

FunctionalEncoding 3.3.2 Continuous

allows a robot to havean infinite spaceof potentialreactions Continuousresponse set that to its world. Insteadof having an enumerated of responses the discretizes way in which therobotcanmove(e.g., (forward, backward left, , . function transforms the , , right , speedupslowdown . . . , etc. }), a mathematical . for into a behavioralreaction Oneof the mostcommonmethods sensory input is implementingcontinuousresponse basedon a techniquereferredto as the . potential fields method ( Wewill revisit why the word basedis italicized in the after we understand potentialfields methodin more the previoussentence detail.) the Khatib ( 1985 andKrogh ( 1984 developed potentialfields methodology ) ) smoothtrajectoriesfor both mobile and manipulator as a basisfor generating a robotic systems This method generates field representinga navigational . basedon an arbitrary potential function. The classic function used is space ' attraction or analogouslythe law of universal that of Coulombs electrostatic , , , gravitation wherethe potentialforce dropsoff with the squareof the distance . betweenthe robot and objects within its environment Goals are treatedas fields areconstructed are . attractors obstacles treatedasrepulsorsSeparate and , the functions to represent relationshipbetweenthe robot based , uponpotential . andeachof the objectswithin the robot' s sensory range Thesefields are then combined typically through superpositioningto yield a single global field. , , For path planning a smoothtrajectorycan then be computedbasedupon the , within the globally computedfield. A detailed presentationof the gradient in . potentialfields methodappears Latombe1991

RobotBehavior

-square expressing relationship law the between forceand Usingtheinverse

distance,
1 Force <X Distance 2

we consttuct a field for a repulsiveobstacle shownin (A ) in figure 3.16. A , ballistic goal attraction field, where the magnitudeof attraction is constant , throughoutspace is depictedfor an attractorlocatedin the lower right of (B) in figure 3.16. In figure 3.17, (A ) showsthe linear superposition thesetwo of fields, with (B) illustrating anexampletrajectoryfor a robot movingwithin this simpleworld. Because a potentialfields encode continuous navigationalspace throughthe world (i .e., the force can be computedat any location they provide ), , sensed an infinite set of possibilities for reaction Potential fields are not without . their problems however (Koren and Borenstein1991 . In particular they are , , , ) vulnerableto local minima (locationswherethe robot may get stuck or cyclic) . methods havebeendeveloped address to these oscillatorybehavior Numerous , problems including the use of hannonic potential fields (Kim and Khosla 1992 Connolly and Gropen 1993 time-varying potential fields (Tianmiao ; ), andBo 1992 randomnoiseinjectedinto the field (Arkin 1987a andadaptive ), ) methods(Clark, Arkin , andRam 1992 amongothers We will examinemore . ), for closelythe methods combiningpotentialfields in section3.4. Another seeminglysignificantproblem with the use of the potential fields methodis the amountof time requiredto computethe entire field. Reactive robotic systems eliminatethis problemby computingeachfield' s contribution at the instantaneous positionmerelywherethe robot is currentlylocated(Arkin 1989b. This is why we saidearlierthat thesetechniques only basedon the are ) fields method no field is computedat all. No path planningis conducted : potential at all: Rather the robot' s reactionto its environment recomputed is as , fast assensory will in processing permit. Oneof the major misconceptions understanding reactivemethods based potentialfields is a failure to recognize on the fact that the only computationneeded that requiredto assess forces is the from the robot' s currentposition within the world. This methodis thus inherently . very fast to computeas well as highly parallelizable When the entire ' field is represented a figure, it is only for the readers edification Reiterating in . , behaviorbasedreactivesystemsusing potential fields do not generate ' plansbasedon the entire field but insteadreactonly to the robot s egocentric of perceptions the world. This is of particular importancewhen the world is , dynamic(i.e., thereare moving objects thus invalidatingstaticplanningtech

100

Chapter3

. . . . " . . . . . . . . . . . . . . . . . . . . : . " " " : . : . " " " " " " . . . . . . . . . . . . . . . . . : . " . . . . . . . . . . . . . . . . . . . " . . . , . . . . . " " " . . " , . . .

. .

. .

. . .

. .

. .

. .

. . .

. .

. . .

. .

. . .

. . . ,

. .

. .

. . .

. .

. . .

. . .

. .

. .

. . . . . . , .

. . . . . . . " " . "

. . . . . . . . . . , , " " . . . , " . . . . . . . . . . . . " . " . . . . . : . . ' " " . . " . . . . . : . . " . . . . . . . . . . . . . . . . : : . . . . . . . . . " . . . . . . . . . : . . . . . . . . . . . . .

. . " " " . "

. . . ,

. . " " . , "

. . . . " "

. , , tt t ttt . .

. .

. ,

. ,

. "

. , , " " .

, ,

, , " ' ~ ~ '

" " " t " l'

" , " -.

" .1 ,

. . . . . . . .

: . "

: .

: "

, , .1 .1 , " ~ . " " ~ " ' \ ' " ~ " " ~ ~ ' ~ . .. .. ... .. ~ .. ~ tfff .. .. ... ..., ,;r . . ~ ~ ~ ~ . .. ... ~ ... ' * .. ~ .. ' '* " ~ ~ ~ Ai A " ~ t . .. ~ ~ . \ I ~ ... - ~ ~ ... .. ~ ~ . .. . . + - ~ ~ ... ... . _ ~ ~ ~ ... ~ ~ ~ ~mm - ~ - -~ ~ - ~- ~- ~ ~ ~ . . 4- + - + . . ... ~ ~ - ~ ~ ~ ~ ~- ~- ~ ~ ~ ~ ~ ~ ~ . ~ .. - ~ ... .. ~ ~ 7 ~ ~ ~ ~ : : : : 7 / : : : : ~ ~~ ~ : : : : ~ : ~ : , .. .. .m .. .. ~ . , m .. .. . . .. .. ~ , iif ~ ~ J ~ " " " , ~" .. ~ . ~ , \ ~ J JJi " ~ " " " ,, ' ' ~ " \ JJ ~ ~ \ " ~ " " ' .. " JJ ~ \ ~ J ~ ~ . " ' ' ~ ' .. " \ \ JJl ~ ~ ~ " " ' " ( \ ( 144 ~ ~ ~ \ " " " " ( ( I " ~ ~ ~ \ " " " \ " \ " I ' ~ ~ ~ \ . . . . . , . , \ , I I , , , , , . . . . . . . . , . , , . . . . , , . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . , ,

-.

ttt1

.1

: .

: . "

: .

(A )

Figure 3.16

on ) readingsare noisy or unreliable(as a plan generated bad Diques or sensor data is likely to be defectiveitself) . Frequentresamplingof the world helps . overcome manyof thesedeficiences Slack( 1990 developed anothermethodfor describingcontinuous ) pathways . ( ) spaceNavigationaltemplatesNATs aredefinedasarbitrary throughnavigational have functionsto characterize navigationalspacethat do not necessarily . any correlation with typical potential fields methods TheseNAT primitives rather characterize spaceon a task orientedapproachand are definedon an . as neededbasis An exampleof this method involves the use of spin-based , techniquesfor obstacletreatment circumventingthe problem of local min-

101

ima found in traditional potentialfields methods In figure 3.18, (A ) showsa . -basedversionfor obstacleavoidance the situationdepictedin (B) in for spin this figure 3.16. The resultsof superpositioning new template(as opposedto field) with (A ) in Figure3.16areshownin ( B) in Figure3.18. By usingknowledge of the goal' s locationrelativeto the detected obstacle a spin direction is , chosen either clockwise(whenthe obstacleis to the right of the goal, viewed , from above or counterclockwise when it is to the left). Unfortunately the ) ( , of the behavioris somewhat with this technique for , modularity compromised
) Figure 3.16 (continued Potentialfields for (A ) an obstacleand ( B) a goal locatedin the lower right sideof the figure.
" " t t , " ? . ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ " \ , t t " ' ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ " \ , t t " ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ , ~ , \ t t " ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ , ~ , , t r , ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ " ~ " t , ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ , ~ \ t , ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ , , \ , ~ ~ ~ ~ ~ t ~ ~ ~ , ~ ~ ~ . . . . . . . . . . _ _ . _ _ . . . . . ~ ~ ~ ~ . . . J \ . . . . . . . ' " ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ~ ~ ' I I ' ~ ~ " " ~ " " " " ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ~ " " II ' J ~ \ . . . . . . . ~ . . . ' " " " ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' . . . ' 11 ' . J ~ ~ \ . . . . . . . ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . " " 11 ' . J ~ ~ \ ~ . . . . . . . . ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ' I I ' . J J ~ ~ \ ~ . . . . . . . . . . . ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ' JiII J ~ ~ ~ ~ ~ ~ . . . . . . . . . . ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . " J J ~ ~ \ \ ~ ~ . . . . . . . . . . ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . " J J ~ ~ \ \ \ ~ . . . . . . . . . . . . . . . . . . ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ' J J J ~ ~ \ \ \ ~ . . . . . . . . . . . . . . . ~ ~ . . . ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ' J J J ~ ~ ~ \ ~ \ ~ ~ . . . . . . . . . . . . ~ . . . ~ . . . . . . ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . J J J ~ ~ ~ \ \ ~ \ ~ ~ . . . . . . . . . . . . . . . . . . ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . J J J J ~ ~ ~ ~ \ \ \ ~ ~ . . . . . . . . . . . . . . . . ~ ~ ~ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . J J J ~ ~ ~ ~ \ \ ~ \ ~ . . . . . . . . . . . . . . . . . . . ~ ~ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . J J J J ~ ~ ~ \ \ \ \ \ ~ ~ . . . . . . . . . . . . . . . . . . . . . ~ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . J J J ~ ~ ~ ~ \ ~ ~ \ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . ~ ~ ~ ~ ~ . . . . . . . . . . . . . . . . . . J J J J ~ ~ ~ ~ \ \ \ \ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . ~ ~ ~ ~ ~ ~ . . . . . . . . . . . . . . . J J J ~ ~ ~ \ \ ~ \ ~ \ \ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . ~ ~ ~ ~ ~ ~ ~ . . . . . . . . . J J J J ~ ~ ~ ~ \ \ \ ~ \ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . ~ ~ ~ ~ ~ ~ ~ ~ ~ J J J ~ ~ ~ ~ ~ ~ \ ~ \ \ ~ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . ~ ~ ~ ~ ~ ~ ~ J J J ~ ~ ~ ~ ~ \ \ \ \ ~ \ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . ~ ~ . . . ~ ~ ~ ~ ~ ~ J J J ~ ~ ~ ~ ~ \ \ \ \ \ \ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ~ . . . ~ ~ ~ ~ J J J ~ ~ ~ ~ ~ \ \ \ \ ~ \ ~ \ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ~ ~ ~ ~ J J ~ ~ ~ ~ ~ ~ \ ~ \ \ \ \ \ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ~ ~ ~ ~ J J J ~ ~ ~ ~ ~ ~ \ \ \ ~ ~ \ \ ~ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ~ ~ ~ JJ ~ ~ ~ ~ ~ ~ ~ ~ \ \ \ \ \ ~ ~ ~ ~ ~ " " " " " " " " " " " " " " " " " " ~ JJ ~ ~ ~ ~ ~ ~ ~ ~ \ \ \ ~ \ \ ~ ~ ~ ~ " " " " " " " " " " " " " " " " " " " " " " J J ~ ~ ~ ~ ~ ~ ~ \ \ \ \ ~ \ \ ~ ~ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . JJ ~ ~ ~ ~ ~ ~ ~ ~ \ ~ \ \ \ \ \ ~ ~ ~ ~ ~ ' " " " " " " " " " " " " " " " " " " J J ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ \ \ ~ \ \ \ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . JJ ~ ~ ~ ~ ~ ~ ~ \ \ ~ \ \ ~ \ \ \ ~ ~ ~ ~ ~ ' " " " " " " " " " " " " " " " " JJ ~ ~ ~ ~ ~ ~ ~ ~ \ ~ \ \ ~ \ \ \ ~ ~ ~ ~ ~ ~ ' " " " " " " " " " " " " JJ ~ ~ ~ ~ ~ ~ ~ ~ \ ~ ~ \ \ \ \ \ \ ~ ~ ~ ~ ~ ~ ' " " " " " " " " " " " " " J J ~ ~ ~ ~ ~ ~ ~ ~ ~ \ \ \ \ ~ \ \ \ ~ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . " ~ J . ' ~ . I ~ . I ~ . I ~ . I ~ . . \ ~ ~ ~ \ ~ \ \ \ ~ \ \ \ \ ~ ~ ~ ~ ~ . . . . . . . . . . . . ~ ~ \ \ \ \ \ ~ ~ ~ \ ~ \ ~ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . , J ~ . " ~ ~ ~ ~ . . . . . 1111 ~ ~ ~ ~ ~ \ ~ ~ \ ~ ~ \ \ \ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ \ \ \ \ ~ \ \ \ ~ ~ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . " ~ ~ ~ ~ ~ ~ . . . . . . IIIII ~ ~ . . " ~ ~ ~ \ ~ ~ \ \ ~ \ \ \ \ ~ ~ ~ ~ ~ ~ . . . . . . . . . . . . ~ \ ~ \ \ \ \ \ ~ ~ \ \ \ ~ ~ ~ ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . " J ~ ~ ~ ~ ~ . . . . . . 11111 ~ ~ ~ ~ ~ \ ~ \ \ \ ~ ~ \ \ \ ~ \ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ \ ~ ~ \ \ \ ~ \ ~ \ ~ ~ ~ ~ ~ ~ ~ . . . . . . . . . . . . " " " " J J ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ \ \ \ \ ~ \ \ \ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ . " J ~ ~ ~ . . " ~ ~ ~ . . . . 111 ~ ~ ~ ~ ~ ~ \ \ \ \ \ ~ ~ \ \ \ \ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ \ \ \ \ ~ ~ \ \ \ \ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ . , J ~ ' ~ . I ~ . I ~ . I ~ . I ~ . I ~ . I ~ . I ~ . . \ ~ . \ ~ ~ \ \ \ \ \ ~ ~ \ \ \ \ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ \ \ \ \ ~ ~ ~ \ \ \ \ ~ ~ ~ ~ ~ ~ ~ . , ~ ~ ' ~ . I ~ . I ~ . I ~ . I ~ . I ~ . I ~ . \ ~ . . \ ~ . \ ~ ~ ~ \ \ \ \ \ \ \ ~ \ \ \ \ ~ ~ ~ ~ ~ ~ ~ ~ \ \ \ \ \ \ ~ ~ \ \ \ \ ~ ~ ~ ~ ~ ~ J ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ \ \ \ \ \ \ \ \ \ \ \ \ \ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ \ \ \ \ \ \ \ \ \ \ \ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ \ \ \ \ \ \ ~ \ \ \ \ \ \ ~ ~ ~ ~ . ' I . I . I . I . I . I . I . I . I . . \ . \ ~ ~ ~ \ \ \ \ ~ ~ ~ ~ ~ \ ~ ~ ~ ~ ~ ~ ~

(B)
' . . . . . . . . . . . . . . . . . . ~ ~ ~ ~ . . . . . . " " . . . . . . . . " " ~ ~ ~ ~ ~

Robot Behavior

102

Chapter3

3.17 Figure

' the correctchoicefor the obstacles spin direction requiresknowledgeof the . , goal location which normally is usedonly within the goal attractionbehavior usesa techniquecalled deformationzones One other novel representation for collision avoidance ) (Zapataet al. 1991 . This techniquedefinestwo manifolds the : the information boundary manifold, which represents maximum from the readingsof all extentof sensing andthe rangemanifold, constructed , activesensorswhich canbe viewedasa deformedversionof the information , of of (figure 3.19 . Perception ) boundarymanifold because the presence obstacles deformationof the information boundarymanifold, andcontrol produces . in to are strategies generated response removethe deformation In the example

'-1"0 I'~'o l"\\'-to -~ 'V oJ 'ra o ~ ~ 1~ 0 0~ ..,O "1 -:rr 0 ",-~ -6 :t'6 ,~~ ",'."I" \~ "I o .\..'.\I+ I\~ ..I.> ,-J, ,1 [~ 6 l -'t. '.~ \' "& J ~ ~ l " A ( )

103

Robot Behavior

( B)

) Figure 3.17 (continued of (A ) Linear superposition fields in figure 3.16. (B) An exampletrajectorythroughthis . navigationalspace shown, the robot would steer to the left . In this approach, reactive behaviors are defined, including emergency stop, dynamic collision avoidance, displacement (orientation ), and target following . One interesting aspect of this work allows for the definition of a variable collision avoidance zone dependent on ' the robot s velocity (i .e., when the robot is moving faster it projects further into the information -boundary manifold , and when it moves more slowly the information -boundary manifold shrinks ) . This enables effective control of the ' robot s speed by recognizing that if the deformation zone cannot be restored by steering, it can be restored by slowing the vehicle down.

104

Chapter3

. . . . .. ....................... . . ". . " ". " " . . . . . . ". . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -. . . . . . . . . . . . . . . . . . . . . . _ -. ~ ~ ~ ~ ~ . . . . ~ . . . . . . . . . . . . . . . . ~~- - - - . . . . . . . . . . . . ~ ~ ..... . ... ... ... ... . , , ~ . . . . . . . . . . . ~ . . . . . . . . ". , ~ , , ..................- ............~..........~ . . . . . . . . . . ~ . . . " " , ........~.. _ - ... ... .. .. ....~ . . . . . . ~~ ~ . . . . . . ", , , , . . , ', ~~~~ ~-+~ .... ..., .......~ . ~ . . . . . . . . . . . "" =~ "-~ " -.~, \ ... \ . . . . . . . . . . . . . . . . . ". "", 1 ,~ ~ . . . ", , " , , ~~ ...\ .. \ . . . . . . . . . . . " " ;~ ~ ~~ '-+~ ~ ~~ \ \ \ . . . . . . . . . . . ~ ~. . ". . ". "" , t t t , ~ ~ ~ ~ ~ ~ ' \ ~~ ~ ", . ". . ". ~ ~, . . . . . . . . , , t t' t~ ~ ~ t~ ~ ~ ~ : ~ ~ ~ ~ ' ' ' :' ' ' ' .:. : .: .: .:t:t~ t tt ". . ". . t , , t ' ~ t t " 't ~t ~~~,/I i } ~ i JJl ; "" : "" : "': : , ~ 1 : ~ , . tttt , . ". . ". , "' , ' , ~ ' ~ ~ ~~ ~ ~ JJ4 "" ", ' "" ~ ~ J4 " ' " . . ". "" "" -, '\ ". . . . " " ' ~ ~ ' "Ji I " " " ,~ ~""~ . . . . . . . . , , , , , , ,~~ ~-'*"-~~ .... ~~..../ I ". , , '. . ". . . . :- ~ .... --.. ~ , . . . . . . . , , , -.-, .. ...+ -+ .... ~ .. - , , , , . . . . . .. ~ ~ -.+ -. . . . . . . """ "", "".."- -- -- r~...~..~ - ..""" "". . . . . . . . . . . . . " , .., .. ..........- ............... . ~ , . . . . . . . . . . . " .. . .. ~ ~ ~ ~ ....~ . ". . . . . . . . . . . . . . . "". . . . . . . ~ ~ . ~ . . . . . . . . . . . . . ~. . ..... . .. .. .... . .. ~ ~ ~ . ... . .. . .. .. . . . . . . . . . . . . . . . . . . . . -. . . . . . . . . . . . . . . . . . . . . . . . . . . -. -- . -. . . . . . . . . . . . . . . . . -. -. . . . . . . . . (A) 3 .18 Figure
3.4 ASSEMBLING BEHAVIORS
Having discussed methods to describe individual behaviors, we now study methods for constructing systems consisting of multiple behaviors. This study requires the introduction of notational formalisms that we will use throughout this book and an understanding of the different methods for combining and coordinating multiple behavioral activity streams. We first examine, however, the somewhat controversial notion of emergent behavior.

105

RobotBehavior

(B)

) Figure3.18(continued of . in (A) NATspinfieldfor theobstacle (B) in figure3.16 (B) Superposition fields(A) . on in thisfigureandfigure3.16bAlsosuperimposed thisfigureis thepathfor a robot . in start field this using combined for thesame andgoalasshown (B) in figure3.17

''"AlJ '..-\~ ~+ \"\l~ \p ~ ,",1l ,,.'\~A ~" i '-I\~ .~I+J ,!~ ,, .lA -" -~ ~+ ~~ "& :.-J ,-;J rl" ~"t+ -/"f'. '~, i:.,~ (1 t~ .A \' _ ~ -' _ " " A AA

Behavior 3.4.1 Emergent

is regardingthe capabilities Emergence often invokedin an almostmysticalsense -based behaviorimplies a holistic capability . of behavior systemsEmergent . than its parts It is true that what occurs where the sum is consider ably greater ' in a behaviorbasedsystemis often a surpriseto the systems designer , of come because a shortcomingof the analysisof the but doesthe surprise of constituentbehavioralbuilding blocks and their coordination or because , else ? something

106

Chapter3

Motion
~
Range Manifold

3.19 Figure
The useof manifoldsfor reactivecollision avoidance .

The notion of emergence a mystical phenomenon as needsto be dispelled , but the concept in a well-defined sensecan still be useful. Numerousresearchers havediscussed in : emergence its variousaspects " . Emergenceis "the appearance novel properties in whole systems of (Moravec1988 . ) . " Global functionality emerges from the parallel interactionof local behaviors " Steels1990 . ( ) " . " Intelligenceemerges from the interactionof the components the system of 's , , , (where the system functionality i.e., planning perception mobility, etc., resultsfrom the behaviorgenerating ) (Brooks 1991a. ) components . " Emergent arisesby virtue of interactionbetweencomponents functionality not themselves with theparticularfunction in mind." ( McFarland and designed Bosser1993 . ) Thecommonthreadthroughall thesestatements thatemergence a property is is of a collection of interactingcomponentsin our casebehaviors The question . , ariseshowever Individual behaviorsare well defined functionally so why : , shoulda coordinated collectionof themproducenovelor unanticipated results ?

Manifold

Boundary

Obstacle

"

Robot

Information

II

, Dire

107

Robot Behavior

Coordinationfunctionsas definedin this chapterare algorithmsand hence contain no surprises and possess magicalperspectiveIn somecases they no . , are straightforward such as choosingthe highest ranked or most dominant , behavior in othersthey may be more complex involving fusion of multiple ; , active behaviors Nonethelessthey are generallydeterministicfunctions and . , . arise certainly computableWhy then doesthe ineffablequality of emergence whenthesebehaviorbased robotic systems released the world? Why can are in we not predict their behaviorexactly ? The answerto this questionlies not within the robot itself but ratherin the . relationshipthe robot has with its environment For most situationsin which the behaviorbased is applied the world itself resistsanalyticalmodeling , paradigm . Nondetenninism rampant Thereal world is filled with uncertaintyand is . . Further the perceptionprocess itself is also poorly char , dynamicproperties acterized Precise : sensor modelsfor openworldsdo not exist. If a world model could be createdthat accuratelycapturedall of its propertiesthen emergence would not exist: Accuratepredictionscould be made But it is the natureof . the world to resistjust suchcharacterizationhencewe cannotpredict apri , ori, with any degreeof confidence in all but the simplestof worlds, how the , world will presentitself. Probabilistic modelscan provide guidancebut not . certainty For example simply considerall the thingsinvolvedin something simple as , as your attendinga class Lighting conditionsand weatherwill affect perceptual . can , processingthe traffic on the roadandsidewalks be characterized only of of weakly (i.e., you do not know ahead time the locationandspeeds all people and cars that you meet . The complexitiesof the world resist modeling ) , " ) leading to the aphorismespoused Brooks ( 1989b: The world is its own by best model." Sincethe world cannotbe faithfully modeledand is full of surprises itself, it is small wonderthat behaviorbasedsystemswhich are tightly , to the world, reflect thesesurprises their designers observers to and grounded or unanticipated actions(or by definition actionsnot by perfonningunexpected ' in ) explicitly captured the designers intentions . inbehavior, Summarizing emergent propertiesare a commonphenomena basedsystems but there is nothing mystical about them. They are a consequence , underlying the complexity of the world in which the robotic agent residesand the additional complexity of perceiving that world. Let us now moveon to methodsfor expressing coordinationof behaviorwithin these the robotic systems .

108

Chapter3

3.4.2 Notation
active Wenow considerthecasewheremultiple behaviors maybe concurrently . within a robotic system Defining additionalnotation let , . S denotea vectorof all stimuli Si relevantfor eachbehaviorPi detectable at timet . . B denotea vectorof all activebehaviors.Biat a giventime t . . G denotea vector encodingthe relative strengthor gain gi of eachactive behaviorPi. . R denotea vectorof all responses generated the setof activebehaviors . ri by A newbehavioralcoordinationfunction, C, is now definedsuchthat

( ( p= CG* B8 , or alternatively ( ) p=CG* R where rl R= r2 . rn

81 gland = s 2 g2B ., = gn
G=

and where * denotes the special scaling operation for multiplication of each scalar component (gi ) by the corresponding magnitude of the component vector (ri ), resulting in a column vector ~ of the same dimension as ri . In other words , ~ represents a response reaction in the same direction as ri with its magnitude scaled by gi . (Alternatively , G can be represented as a diagonal matrix ). Restating, the coordination function C , operating over all active behaviors B , modulated by the relative strengths of each behavior specified by the gain vector G , for a given set of detected stimuli at time t , S, produces the overall robotic response p , where p is the vector encoding the global response that the robot will undertake, representedin the same form as r (e.g ., [ x , y , Z, (J, < , 1 ]) . /> /1 C can be arbitrarily defined, but several strategies are commonly used to encode this function . They are split across two dimensions: competitive and cooperative. The simplest competitive method is pure arbitration , where only ' one behavior s output (ri ) is selected from R and assignedto p , that is , arbitrar ily choosing only one responsefrom the many available. Several methods have

1 .n 8
,

109

.RobotRehavinr been used to implement this particular technique, including behavioral priori tization ( subsumption) or action selection. Cooperative methods, on the other hand, blend the outputs of multiple behaviors in some way consistent with the ' agent s overall goals. Th ~ most common method of this type is vector addition . Competitive and cooperative methods can be composed as well . Section 3.4.3 examines these methods in more detail . First , however, let us revisit the classroom example. Given the robot ' s current perceptions at time t : (class - location , 1.0) (detected - object , 0.2) (detected - student, 0.8) (detected - path , 1.0) (detected - elder , 0.0 )

S=

for each Si = (p , A), where A is the stimulus p ' s percentage of maximum strength. The situation above indicates that the robot knows exactly where the classroom is , has detected an object a good distance away, sees a student approaching nearby, sees the sidewalk ( path) with certainty , and senses that there are no elders nearby. We can then represent the behavioral response as: - to .8move - classs 1 ( ) - object ) (S2 .8avoid - student ) (S3 . 8dodge ( ) . 8stayright S4 ( ) .8de - to- elderSS / er

B (8 ) =

R then is computed using each .8,

R=

with component vector magnitudes equal to (arbitrarily for this case)

1 . 0 0 . 8 1 0

110

Chapter3

where each ri encodesan [x , y , 6] for this particular robot expressingthe . behavior In the example for desireddirectional response eachindependent , -object anddefer to-elder arebelow thresholdandgenerate response no avoid , whereas moveto-classandstay-right areat maximumstrength(definedas 1.0 is from ,8: S -.. R. that here . Remember the response computed ) Before the coordinationfunction C is applied R is multiplied by the gain , vector G. For the numbersused in the example for G, stay-right is least important (gstay right = 0.4), dodgestudent is deemedthe most important = - student 1.5), while moveto-classanddefer to-elder are of behavior(gdodge = equal intermediatepriority (gmoveto- class gde er- to- elder= 0.8) and the ! = overall (gavoidobject 1.2). avoid-object behaviorrankssecond R' = G * R , where :

gmove to- class gavoid- object

G=

gdodge student - right gstay gdef er - elder

: yielding

g. * r . g2* r2 = C(R' ) = C p g3* r3 g4* r4 . gs* rs

vector with scaled magnitudes component

' '_...1 R magm - ,-

G is of value only when multiple behaviors are being coordinated , as it enables the robot to set priorities through establishing the relative importance of each of its constituent behaviors. If a simple winner -take-all coordination strategy is in place (action selection), a single component vector Girl ( based on some metric such as

. 0 8 1 2 . 5 4 0 . 8 0 . 8 1 . 2 4 0
I

'--

Robot Behavior

toP executed example . Inthe ) ischosenC, assigned and greatest magnitude by above isthe with of .2 with , this componentavalue 1 , associatedthe -student . Thus response behavior the undertaken this arbitration dodge using simple function action isthe . required tododgeoncoming .Note an student that this behavior dominates given current only the perceptual attime readings t and the that behavioral willchange robotperceptions as the 's ofthe response -based -ranked world change apriority arbitration , the also . For system highest behavior inG threshold be and component encoded (above ) would chosen executed ofthe . Inour , independentindividual componentsrelativemagnitudes would be , dodge , assuming example -student also chosen this using method -ranked the stimulus threshold gdodge (I.5) isthe isabove , since - student highest behavior . functionsrecursively tooperate only defined not , coordination are Finally -level on low behavioral but also the of coordination responses on outputother (which are operators also behavioral ). responses PI , ' P2 , P= C . Pn
where p ' coordinatesthe output of lower-level coordinativefunctions One . benefitof this definition is the ability to sequence of behavioralactivities sets for a robot temporally .
3.4. 3 Behavioral Coordination

We now turn to examinethe natureof C, the coordinationfunction forbehavlors. This function hastwo predominant classescompetitiveandcooperative , , each of which has severaldifferent strategiesfor realization The different . classes , may be composed together althoughfrequentlyin the behaviorbased architectures described chapter4, a commitmentis madeto one type of coordination in function specificto a particularapproach .

~ 3.4.3 Competitive Methnd .1

Conflict canresultwhentwo or morebehaviors active eachwith its own independent are , . Competitivemethods of response providea means coordinatingbehavioral for . response conflict resolution The coordinatorcan often be viewed

112
p e r c e p t i 0 n

Chapter3

Response of highest active behavior

Figure 3.20 network . Arbitration via suppression

for as a winner-take all networkin which the singleresponse the winning behavior all . out-muscles the othersandis directedto the robotfor executionThis canbe enacted a variety of ways. in type of competitivestrategy Arbitration requiresthat a coordinationfunction servingasan arbiter select . a single behavioralresponseThe arbitrationfunction can take the form of a fixed prioritization network in which a strict behavioraldominance hierarchy and exists typically throughthe useof suppression inhibition in a descending , -basedmethods(Brooks 1986 the mannerThis is a hallmark of subsumption . ), in particularsof which aredescribed chapter4. Figure 3.20 showsan example . illustratedasa dominance hierarchy Action-selectionmethods(Maes 1990 arbitrarily selectthe outputof a single ) . behavior but this is donein a less autocraticmanner Here the behaviors , actively competewith eachother through the use of activationlevels driven ' ) by both the agents goals (or intentionality and incoming sensoryinformation es . No fixed hierarchyis establishedratherthe behavioral ; process compete with eachother for control at a given time. Run time arbitrationoccursby the . selectionof the most active behavior but no predefinedhierarchyis present , . Figure 3.21 capturesthis notionally The conceptof lateral inhibition (section ' 2.2.1) caneasilybe implemented using this methodwhereonebehaviors 's strongoutputnegativelyaffectsanother output. ' Anotherevenmoredemocratic competitivemethodinvolvesbehaviors generating votes for actions with the action that receivesthe most votes being , and the single behaviorchosen( Rosenblatt Payton 1989 . This technique )

&

_ .

. II

Priority based Coordination

113
Q

RobotBehavior

ActionSelection Coordination

Figure3.21 Arbitration actinn - ~ 1ect1nn via

, is embodied in the Distributed Architecture for Mobile Navigation (DAMN ) ' (Rosenblatt 1995) . Here instead of each behavior s being encoded as a set of rule -based responses each behavior casts a number of votes toward a predefined , set of discrete motor responses For navigation of an unmanned ground . vehicle , the behavioral response set for steering consists of {hard - left , soft -left , straight -ahead , soft - right , hard -right } . Each active behavior (e.g ., goal -seeking, obstacle-avoidance, road -following , cross-country , teleoperation , map-based navigation ) has a certain number of votes (gi ) and a user-provided distribution for allocating those votes. Arbitration takes place through a winner -take-all strategy in which the single response with the most votes is enacted. In a sense this allows a level of behavioral cooperation , , but the method is still arbitrary in its choice of a single response. Figure 3.22 depicts this type of arbitration method.

3.4.3.2 Cooperative ethods M

Cooperative methods provide an alternative to competitive methods such as arbitration . Behavioral fusion provides the ability to use concurrently the output of more than one behavior at a time . The central issue in combining the outputs of behaviors is finding a representation amenable to fusion . The potential - fields method, as described in section 3.3.2, provides one useful formalism . As shown earlier , the most straightforward method is through vector addition or superpositioning . Each behav' ior s relative strength or gain , gi , is used as a multiplier of the vectors before addition . Figure 3.23 illustrates how this is accomplished.

with

level

behavior

of

activation

hlgh881

Response

B4

act

>

B3

act

'

B2

act

act

MAX

= R I . . L

114

Chapter3

Voting based Coordination

3.22 Figure
Arbitration via voting.

p e r c e p t i 0 n

Fused behavioral response

3.23 Figure

. Behavioralfusion via vector summation

and Figure 3.24 showsa field with three active obstacles a goal locatedin . the lower right comer In (A ), the goal attractorbehaviorhas twice the obstacle ' . behaviors relativestrength The paththe robot takesthrough avoidance themselvesresultingin a shorteralthough this field cuts fairly closeto the obstacles , has avoidance twice the strength . In (B), theobstacle moreperilouspath . of the goal attraction The path taken is consider ably longer but also further . themselvesVectoradditionprovidesa continuumfor comfrom the obstacles
biDing these fields controllable In work at Hughes Artificial of combining multiple through the gain vector G . Intelligence Center (Payton et al . 1992 ) , the issue disparate .behavioral outputs is handled by avoiding

of

with

votes

Response

behavior

most

, votes

Rs

, votes

, votes

>

R4

votes

votes

MAX

p r p i e c e ~ t

115

RobotBehavior the sole use of single discrete response values. Instead, the responses of each behavior constrain the control variables as follows : . Zone: establishment of upper and lower bounds for a control variable . Spike : single value de ignation resulting in standard priority -based arbitration ~ . Clamp : establishment of upper or lower bound on a control variable In essence the system provides constraints within which the control system , must operate to satisfy its behavioral requirements. The algorithm for fusing the behaviors is as follows : ' Generate a profile by summing each active behavior s zones and spikes If maximum is a spike choose that command else (maximum is in zone ) minimal change from current choose commandrequiring control setting If command chosen beyond a clamped value choose most dominant clamp from all clamps If that clamp dominates profile move commandvalue to just within clamped region In this manner, commands can be described in the control variables for each actuator (e.g ., speed of a motor , angles of a control surface, etc) . Clamps limit the acceptable ranges for each actuator. This particular system has been applied to the control of an underwater robot using high - level behaviors such as stealth, safety, urgency, and efficiency , which are mapped onto low - level behaviors that ' control the vehicle s speed, the angle of the dive plane, proximity to the ocean bottom , and so forth , which in turn control the vehicle actuators, such as the ballast pumps and dive plane motor. Formal methods for expressing behavioral blending are discussed in Saffiotti , Konolige , and Ruspini 1995. In this approach, each behavior is assigned a desirability function that can be combined using specialized logical operators called t - norms, a form of multivalued logic . A single action is chosen from a set of actions created by blending the weighted primitive action behaviors. As this is essentially a variation of fuzzy control , where blending is the fuzzifi cation operation and selection is the defuzzification process, the method for generating these results is deferred until chapter 8.

116

3.4.4

Behavioral Assemblages

,-"\..I .-'t/'~ .os ~ ~ S ,\.i."" .J . "~I "\r~ ''_I, J ,S ,~ ~a +\l+1 a~ \r\""\A ,_ '"_'Ir "'\" ,"A ..',A S -~ + , ":.I.t, . J ,-, "\',ig .+ ~4 1 1 .~~ ~ --a _ + ~ ~ ~ A ( ) . 2 F
Chapter3

It is also possiblethat the deformationmanifolds approachdescribedfor obstacleavoidancein section 3.3.2 could be extendedto behavioralfusion as the approach for potentially can provide a commonrepresentation robotic action. .So far, however it hasbeenlimited to arbittatioD. ,

Behavioral assemblagesare the packages from which behavior-based robotic systems are consbucted. An assemblageis recursively defined as a coordinated collection of primitive behaviors or assemblages Each individual assemblage . consists of a coordination operator and any number of behavioral components

117

RobotBehavior

) Figure 3.24 (continued . avoidance dominates Behavioralfusion: (A ) goal attractiondominates ( 8) obstacle ;

",.,, "'-"~ ",'I " 1 '" I .~1 .\'.~ ,'.,,~ e"" e ., 'Ir.,, 0'1 ,"". "~., I ,",\" .\~ "t ,. ~ '. " . ~ ( B )
use ). (primitivesor assemblagesThe powerof assemblage arisesfrom the notion from simpler of abstractionin which we cancreatehigher level behaviors , of their constituent elements . onesandrefer to themhenceforward independent Abstractionhastraditionally beena powerful conceptin AI (Sacerdoti1974 ), in and it is no less so here Abstractionenablesus to reuseassemblages an . ; , easy modularmannerto constructbehaviorbasedsystemsprovidesthe ability 6 to reason overthemfor usein hybrid architectureschapter ); andprovides ( to and coarser levelsof granularityfor adaptation learningmethods be applied (chapter8).

118

Chapter3

Wehavepreviouslyusedthe notationC (G * B (8)) = p, in section3.4.2. We denotean assemblage suchthat qi symbolizesC (G * B (8)), henceqi = p . qi For conveniencewe will generallyomit the p in assemblage , , expressionleaving which mapsconveniently onto a statewithin anFSA diagram Although . qi , can in assemblages be expressed anyof the notationalformatssection3.4.2 describes thereis a temporalcomponent i.e., the behavioralstructure , whenever ( of the systemchanges over time), FSA notationis usedmostoften. of in Figure 3.12 hasalreadyshownan assemblage assemblages FSA form for a competition robot. The constituentbehaviorsfor two of that robot' s statesare assemblage . move to- pole . move to-goal(detectpole) -static obstacledetectobstacles avoid ( ) . noise generate -direction low gain ( ) . wander . probe detectopen area ( ) . avoid static obstacledetectobstacles ( ) . noisegenerate -direction high gain ( ) . avoid past detectvisited-areas ( )
.

Each of theseFSA statescan be equivalentlydepicted as an SR diagram , an in exampleof which appears figure 3.25. All of theseassemblages , in turn, be bundledtogetherin a high-level SR can if desired(figure 3.26). Using this alternate , which has diagram representation label for the coordinationfunction, the explicit temporal merely a sequencer betweenstates(i .e., the state transition function q associated dependencies with the perceptual to ) triggers arelost whencompared the earlierFSA version 3.12 . (figure ) Another form of assemblage construction referredto rather as " hierarchi , " in . , cally mediatedbehaviors appears Kaelbling 1986 A similar hierarchical abstractioncapability also appears teleo in -reactivesystems( Nilsson 1994. ) In RS (Lyons and Arbib 1989 an assemblage defined as a network of is ), schemas can be viewed as a schema that itself. In all thesecases behavioral , ( ) can , aggregationsor assemblages be viewedrecursively(or alternatively hi. themselves ) erarchically asbehaviors behavioral abstractionsconstitute , definedashierarchicalrecursive , Assemblages the primary building blocks of behaviorbasedrobotic systems are and attached sensors actuators to and . ultimately groundedin primitive behaviors

119

RobotBehavior detect-pole

detect -obstacles

r ;

Move -to- pole Response

generate -direction

detect -open -area detect - obstacles

-diredion generate

Wander Response

(8) . Figure32S
. -toSR diagramfor the move pole andwanderassemblages

Overall Competition Behavior

.1

..--.__ _ .

1 /

Figure3 . 26 . CompetitionSR diagram

120

Chapter3

3.5 CHAPTER SUMMARY

. Robotic behaviors generate a motor response from a given perceptual stimulus . . Purely reactive systems avoid the use of explicit representational knowledge . . Behavior -based systems are inherently modular in and provide the design ability for software reuse. . Biological models often serve as the basis for the design of behavior -based rob Otic systems. . Three design paradigms for building behavior -based systems have been presented : ethologically guided/ constrained design, using biological models as the basis for behavioral selection, design and validation ; situated activity , which creates behaviors that fit specific situational contexts in which the robot will need to respond; and experimentally driven design, which uses a bottom -up design strategy based on the need for additional competency as the system is being built . . The expression of behaviors can be accomplished in several different ways : SR diagrams, which intuitively convey the flow of control within abehavior based system; functional notation , which is amenable to the generation of code for implementation ; and FSA diagrams, which are particularly well suited for ' representing behavioral assemblages time -varying composition . . Other, more formal methods have been developed for expressing behaviors, such as RS and situated automata as epitomized by the Gapps language. . Behaviors can be represented as triples ( S, R , .8) , with S being the stimulus domain , R the range of response, and .8 the behavioral mapping between them. . The presence of a stimulus is necessary but not sufficient to evoke a motor response in a behavior-based robot. Only when the stimulus exceeds some threshold value t' will it produce a response. . gi , a strength multiplier or gain value, can be used to turn off behaviors or increase the response' s relative strength. . Responses are encoded in two forms : discrete encoding , in which an enumerable set of responses exists; or continuous functional encoding , in which an infinite space of responsesis possible for a behavior. . Rule -based methods are often used for discrete encoding strategies. . Approach es based on the - fields method are often used for the continuous potential functional encoding of robotic response. . There is nothing magical about emergent behavior ; it is a product of the complexity of the relationship between a robotic agent and the real world that resists analytical modeling .

121

Robot Behavior . Notationalmethodsfor describingassemblages coordinationfunctions and . usedthroughoutthe text havebeenpresented . The two primary mechanisms behavioral coordinationarecompetitiveor for . , but they canbe combinedif desired cooperative . Competitivemethods resultin the selection the outputof a singlebehavior of , . typically eitherby arbitrationor action selection . Cooperative oftenusesuperpositioning forcesor gradients of methods generated es from field- basedmethods including potentialfield-basedapproach or , . navigationaltemplates . Assemblages recursivelydefinedaggregations behaviorsor other as of are se mblages . . . Assemblagesserve as important abstractions useful for constructing . behaviorbasedrobots

Chapter Behavior

4 - Based Architectures

One can expect die human race to continue attempting systems just within or just ; beyondour reach and softwaresystemsare perhapsdie most intricate and complex . . of man' s handiworks The management this complexcraft will demandour bestuse of and of new languages systemsour bestadaptation provenengineering of , management . medlods liberal dosesof commonsense and a God-given humility to recognizeour , , fallibility andlimitations. - FrederickP. Brooks, Jr. . Therearetwo waysof constructinga softwaredesign One way is to makeit so simple that thereareobviouslyno deficienciesAnd the otherway is to makeit socomplicated . . that thereareno obviousdeficiencies - C .AiR. Hoare

to

subsumption :

robotic

robotic

available

based

based

architectures

choices

behavior

behavior

robotic

of

of

architectural

is

design

reactive

based

the

construction

for

the

different

for

architecture

behavior

two

robot

other

requirements

depth

the

principles

schema

of

in

what

the

builder

design

motor

many

and

system

Objectives understand understand develop characterize review robot To To To To To . . . . . 1

architecture

Chapter 2 architecture the 4

124

Chapter4

4.1 WHATIS A ROBOTIC ARCIDTECTURE ?

In chapter3, we learnedabout robotic behaviors including methodsfor expressing , them. To designandbuild behaviorbased , encoding andcoordinating , robotic systemscommitments needto be madeto the actual specificmethods , , to be usedduring this processThis needleadsus to the studyof robotic architectures . : softwaresystems specifications providelanguages tools and that and for the constructionof behaviorbasedsystems . All of the architectures described this chapterare concerned in with behavioral control. As describedin chapter 1, severalnon- behaviorbasedrobotic architecturesappearedbefore the advent of reactive control, for example , NASREM (section 1.3.1). Here however we focus on behaviorbasedsystems , , . Though consider sharemany common , ably varied thesearchitectures features : . emphasis the importance coupling sensing actiontightly on of and . avoidance representational of symbolicknowledge . decomposition into contextually meaningfulunits ( behaviors situation or actionpairs) sharea common philosophy on the surface , Although thesearchitectures therearemanydeepdistinctionsbetweenthem including , . the granularityof behavioraldecomposition . the basisfor behaviorspecification(ethological situatedactivity, or experimental , ) . the response ) encodingmethod(e.g., discreteor continuous . the coordinationmethods used(e.g., competitiveversuscooperative ) . the programmingmethods languagesupport available and the extent of , , softwarereusability . In this chapterwe study severalcommonrobotic architectures usedto build . behaviorbasedsystems Tablesappearthroughoutsummarizingthe characteristics for each of the behaviorbasedarchitectures discussed Two of the . architectures have been singled out for closer scrutiny than the others the : -basedarbitration architecture rule-basedencodings and priority subsumption using combination ; and motor schemas using continuousencodingand cooperative of vectors .

125

Behavior -Based Architectures

4.1.1 Definitions
Perhapsa good place to begin searchingfor our definition of robotic architectures would be with ~ e definition of computerarchitecturesStone( 1980 . , . 3), one of the best known computer architects uses the following definition , p : " Computer architectureis the discipline devoted to the design of from a collectionof commonbuilding highly specificandindividual computers blocks." Roboticarchitectures essentially same In our roboticcontrolcontext are the . , however architecture , , usually refersto a softwarearchitectureratherthan the ' hardwareside of the system So if we modify Stones definition accordingly . , we get: , Roboticarchitecture the is devoted the designof highly specificand to discipline individual robots froma collection common of software blocks . building How doesthis definition coincidewith otherworking definitionsby practicing robotic architects Accordingto HayesRoth ( 1995 p. 330), anarchitecture ? , " . . . the abstract refers to of a class of agents the set of structural : design in which perception reasoning and action occur; the specific , , components , functionality and interfaceof eachcomponent andthe interconnection topol" . is ogy betweencomponents Although her discussionof agentarchitectures for artificially intelligent systems general it alsoholdsfor the subclass in , targeted with which we are concernednamely behaviorbasedrobotic systems . , , Indeed a surveil lancemobilerobot system(figure4.1) hasbeendeveloped that , -Roth et al. 1995 . Sheargues embodies architectural her ) designprinciples( Hayes that architectures mustbe produced fit specificoperating to environments , a conceptcloselyrelatedto our earlierdiscussion ecologicalnichesin chapter of 2 andrelatedto the claim in McFarlandandBosser1993that robotsshould be tailoredto fit particularniches . Mataric ( 1992a providesanotherdefinition, stating "An architecture , ) provides a principled way of organizinga control system However in addition . , to providing structure it imposesconstraintson the way the control problem , canbe solved" Onefinal straightforward . definition is from DeanandWellman " a , ( 1991 p. 462) : An architecturedescribes set of architecturalcomponents . andhow they interact"

126

Chapter4

Figure 4.1 Nomad200 robot of the type used for surveil lance at Stanford ( photograph . . courtesy of NomadicTechnologies , MountainView, California.) Inc.

4.1.2 Computability
are , Existing robotic architectures diverse from the hierarchicalNASREM architectureto purely reactivesystemssuch as subsumption section4.3) to ( architectureschapter6). In what ways can instances chosenfrom the ( hybrid ? diversity of architecturalsolutionsbe saidto differ from one another In what canthey be saidto be the same ? ways The answerto thesequestionsis related to the distinction betweencomputability and organizingprinciples Architecturesare constructed . from components its own peculiarsetof building , with eachspecificarchitecture having blocks. The ways in which thesebuilding blocks can be connected facilitate certaintypesof robotic designin given circumstancesOrganizingprinciples . 's underlie a particular architecture commitmentto its componentstructure , . , andconnectivity granularity From a computationalperspectivehowever we may seethat the various , , architectures all equivalentin their computationalexpressiveness are . Consider between . , for instance the differences , programminglanguagesDifferent

127

Behavior - Based Architectures

to choicesareavailableto the programmer rangingfrom machinelanguage assembler -level languagessuchasFortran Cobol, C, Pascaland to varioushigh , , ( . suchasthoseusedin visual programming LISP) to very high- level languages in the ideathat one language do can Is thereany fundamental incompatibility

that cannot ? something another the Consider resultsthat BohmandJacopini( 1966 derived ) concerning in . Theyproved if anylanguage that languages computability programming
of containsthe threebasicconstructs sequencingconditionalbranching and , , functions(i.e., it is Thring iteration, it cancomputethe entireclassof computable ) equivalent. This essentiallystatesthat from a computationalperspective haveno differences . the commonprogramming languages The logical extensionis that sinceall robotic architectures provide the capability to perform taskssequentially allow conditionalbranching and provide , , are the ability for iterativeconstructsthesearchitectures computationally , are . equivalent All behaviorbasedrobotic architectures essentiallysoftware . or languages frameworksfor specifying and controlling robots The level of . offer may differ, but not the computability abstraction they This does not mean of coursethat we will start writing AI programsin , has Cobol. It doesmeanthat eachcurrentprogramming language in turn found well and thus hassurvived(i.e., remainedin usage a nichein which it serves ) . holds because is well suitedfor thatparticulartask Somearguethat the same it as for robotic architectures well: each servesa particular domain (or niche ) es and will be subjectedto the sameenvironmentalstress for survival as are . or computerarchitectures softwarelanguages Behaviorbasedrobotic systemsservebest when the real world cannotbe can . characterized modeled Whenever or engineering removeuncertainty accurately behaviorbased from the environmentpurely , may not necessar systems ily afford the best solutionfor the task involvedandhierarchicalarchitectures , , , (chapter 1) may prove more suitable as for example in factory floor operations . where the environmentcan be altered to fit the robot' s needs More often than not, however much as we try, we cannotremoveuncertainty unpredictability , , , and noisefrom the world. Behaviorbasedrobotic architectures in to were developed response this difficulty and chooseinsteadto deal with theseissuesfrom the onset relying heavily on sensingwithout constructing , . during global world models The morethe world changes potentiallyerroneous a execution themoretheresultingvalueof anyplangenerated priori decreases , , of andthe moreunstable representational knowledgestoredahead time or any becomes . during executionandremembered gathered

128

Chapter4

At a finer level, we will seethat behaviorbasedarchitecturesbecause of , their differentmeans expressing of behaviors the setsof coordinationfunctions and ' . they afford, provide significantdiversity to a robotic systems designer Eachapproach its own strengths weaknesses termsof what it is best has and in at doing or whereit is mostappropriately . of applied The remainder this chapter discuss a variety of behaviorbasedrobot architecturalsolutions Not all es . are expected withstandthe test of time, and many will likely suffer a fate to similar to that of early programming ( languagese.g., ALGOL , SNOBOL) and fadeoff into obscurity Ecologicalpressure . from sources of rangingfrom ease usefor the designer generalizabilityto public opinion to exogeneous to factors , (political, economic etc.) will ultimately serveas the fundamentalselection 's mechanismnot merely an academic perspective their elegancesimplicity on , , , or utility . As an aside we notethe recentcontroversy Penrose 1989 1994 stirredup , , ( ) that no computerprogramcaneverexhibit intelligenceasaccording by claiming to Penroseintelligencemustincorporate solutionsto noncomputable , problems as well as thosethat are computable Interestingly he doesnot dismiss . , the attainmentof intelligenceas utterly impossiblein a deviceand presents a novel, but rather speculative approachbasedon quantummechanics a microtubule , ( architectureif you will ), rather than a computationalapproach to , achievethis goal. To say the least his position hasbeenstrongly rebuttedby , as . manywithin the AI communityandis often cursorily dismissed rubbish In the book' s final chapter we will revisit this issueof what intelligencemeans , within the contextof a robotic systemandwhat we canor shouldexpectfrom thesesystems Suffice it to say for now, that all the behaviorbasedarchitectures . , considered this book arecomputational in .
4.1. 3 Evaluation Criteria How can we measure an architecture ' s utility for a particular problem ? A list of desiderata for behavior -based architectures is compiled below. . Support for parallelism : Behavior -based systems are inherently parallel in nature. What kind of support does the architecture provide for this capability ? . Hardware targetability : Hardware targetability really refers to two different things . The first regards how well an architecture can be mapped onto real robotic systems, that is , physical sensors and actuators. The second is concerned with the computational processing. Chip -level hardware implementations are often preferred over software from a performance perspective. What

129

BehaviorBasedArchitectQres

type of supportis availableto realizethe architecturaldesignin silicon (e.g., ]) compilersfor programmable logic arrays[Brooks 1987b ? . Niche targetabilty : How well can the robot be tailored to fit its operating environment between robotand (HayesRoth 1995 ? How canthe relationships ) environment expressed ensuresuccessful be to nicheoccupation ? . Support for modularity : What methodsdoes an architectureprovide for behavioralabstractions Modularity can be found at a variety ? encapsulating of levels By providing abstractions use over a wide rangeof behavioral . for ' levels (primitives assemblages , , agents, an architecturemakesa developers ) task easierandfacilatessoftwarereuse(Mataric 1992a. ) . RobustnessA strengthof behaviorbased : is systems their ability to perform in the face of failing components(e.g., sensors actuators etc.) ( paytonet , , doesthe ; ) , al. 1992 Horswilll993a ; Ferrell 1994 . What typesof mechanisms architecture for suchfault tolerance ? provide . nmeliness in development What typesof tools anddevelopment : environments are availableto work within the architecturalframework Is the architecture ? more of a philosophicalapproachor doesit provide specifictools and , methodsfor generating robotic systems real ? . Run time ftexibility : How can the control systembe adjustedor reconfigured ? and ? during execution How easilyis adaptation learningintroduced . Performance effectiveness How well doesthe constructed : robot perform its intendedtask(s)? This aspectalso encompass the notion of timelinessof es execution or how well the systemcan meet established -time deadlines real . , In other instances specificquantitativemetricscan be appliedfor evaluation , ) purposeswithin a specific task context (Balch and Arkin 1994 . Thesemay includesuchthingsastime to taskcompletion energyconsumptionminimum , , travel, and so forth, or combinations thereof . Thesewidely rangingcriteria canbe usedfor evaluatingthe relative merits described the remainder this chapter in of . of manyof the architectures

4.1.4 Organizing Principles

in From the discussion chapter3, severaldifferent dimensions fordistinguishing robotic architectures becomeapparentincluding , . Different coordinationstrategiesof particularnote competitive(e.g., arbitration , , , , action selection voting) versuscooperativee.g., superpositioning ( ) . Granularity of behavior microbehaviorssuch as those found in situated : ) ( activity-basedsystems e.g., Pengi or more generalpurposetaskdescriptions e.g., RAPs . ( )

130

Chapter4 . Encoding of behavioral response: discrete, that is , a prespecified set of possible responses (e.g ., rule -based systems or DAMN ), or continuous (e.g., potential field - based methods) . The remainder of this chapter first discusses two architectures in some detail : the subsumption architecture and motor schema based systems (the reactive of the Autonomous Robot Architecture (AuRA ) . Next it reviews component several other significant behavior-based architectures, although at a higher level. Finally it presents design principles for constructing a behavior -based robotic system, in an architecture- independent manner as much as possible.

4.2 A FORAGING EXAMPLE

To ground the following architecturaldiscussions let us consider a well, studiedproblemin robotic navigation foraging. This taskconsistsof a robot' s : . moving awayfrom a homebasearealooking for attractorobjects Typical applications lost itemsof value . might includelooking for something or gathering , Upon detectingthe attractor the robot movestoward it , picks it up and then returnsit to the homebase It repeats . this sequence actionsuntil it hasreturned of all the attractorsin the environmentThis test domainhasprovidedthe . basisfor a wide rangeof resultson both real robots(Balchet al. 1995 Mataric ; 1993a andin simulation Foragingalsocorrelates . well with ethologicalstudies ) , especiallyin the caseof ants(e.g., Gosset al. 1990 . ) -level behavioralrequirements accomplish task include: Several to this high 1. Wander movethroughthe world in search an attractor : of 2. Acquire: movetowardthe attractorwhendetected 3. Retrieve returnthe attractorto the homebaseonceacquired : thesehigher level assemblages . Eachassemblage shown Figure4.2 represents is manifestedwith different primitive behaviorsand coordinatedin different waysaswe movefrom onearchitectural exampleto the next.

4.3 SUBSUMPTION ARCIDTECTURE

the architecturein the mid- 1980sat RodneyBrooks developed subsumption the Massachusetts Institute of Technology His approach a purely reactive . , behaviorbasedmethod flew in the faceof traditional AI research the time. at , -plan-act paradigm used in some of the first Brooks arguedthat the sense autonomous robots suchas Shakey(Nilsson 1984 was in fact detrimentalto )

131

. BehaviorBased Architectures

Figure4.2 FSAdiagram foraging for . the constructionof real working robots He further arguedthat building world . modelsand reasoningusing explicit symbolic representational knowledgeat best was an impedimentto timely robotic response at worst actually led and roboticsresearchers the wrong direction. in In his seminalpaper Brooks ( 1986 advocated useof a layeredcontrol the , ) architecture layeredalonga different but , systemembodied thesubsumption by dimensionthan what traditional research was pursuing Figure 4.3 showsthe . -plan-act vertical model illustratedin distinction with the conventional sense , in (A ) andthe new horizontaldecomposition (B) . (The orientationof the lines that separate components the determines vertical andhorizontal) . Much of thepresentation styleof the subsumption and is . approach dogmatic Tenets this viewpoint include of . Complexbehaviorneednot necessarily the productof a complexcontrol be . system . Intelligenceis in the eyeof the observer Brooks1991a. ( ) . The world is its own bestmodel(Brooks 1991a. ) . Simplicity is a virtue. . Robotsshouldbe cheap . . Robustness the presence noisy or failing sensors a designgoal. in of is . Planningis just a way of avoiding figuring out what to do next ( Brooks 1987a. ) . All onboardcomputation important is . . Systems shouldbe built incrementally . . No representation No calibration No complex computers No high. . . bandwidthcommunication Brooks1989b. ( )

132

Chapter4

-1 -. 1 -.----~-.--- ...--8 --.J 1 -- 1 ~...... ..----......-...._ -- . -- --- I. 1 -, I , - --- -- I .._~ " ~ ~ ~ ~~-~~C.. I
(A)

~
I . .

. ~

(B )

Figure 4. 3 . ( ) (A ) Senseplan-act model. ( B) Subsumptionreactive model

This was hard to swallow for many in the AI community (and in many casesstill is). Brooks lobbiedlong andhardfor rethinkingthe way intelligent , in . robotsin particular and intelligent systems general shouldbe constructed , , This stance the roboticsresearchAlthough . changed direction of autonomous manyin the AI communitytake a more tempered position regarding currently the role of deliberationand symbolicreasoning chapter6), Brooks hasnot to ( in datedisavowed print any of theseprinciples( 1991a. ) architectureTable4.1 . Let us now moveto the specificsof the subsumption view is the first of manytablesthroughoutthis chapterthat providea snapshot of in of the designcharacteristics a particulararchitecture light of the material in discussed chapter3..
4. 3.1 Behaviors in Subsumption

architectureare represented as Task achievingbehaviorsin the subsumption on individual goals concurrentlyand . separate layers Individual layers work . using an asynchronouslyAt the lowest level, each behavior is represented finite statemachine(AFSM) model(figure4.4). The AFSM encapsulates augmented transformation function.8 . Stimulusor response a particularbehavioral ; . or canbe suppressed inhibited by other activebehaviors A resetinput signals . is alsousedto returnthe behaviorto its startconditions EachAFSM performs

133

Behavior - Based Architectures

Table4.1 Architecture Subsumption Name Background Precursors Prindpal designmethod Developer

Response encoding Coordination method Programming method Robots fielded
References

architecture Subsumption -known early reactivearchitecture Well ; ; Braitenberg1984 Walter 1953 Ashby 1952 Experimental RodneyBrooks ( Mit ) discrete(rule based Predolninantly ) -basedarbitrationvia inhibition Competitive(priority and suppression ) Old methodusesAFSMs; new methodusesBehavior Language Allen, Genghis(hexapod, Squirt (very small), Toto, ) others , Seymour Polly (tour guide several ), Brooks 1986 Brooks 1990b Horswill 1993a ; ;

Reset Suppressor

INPUT WIRES

BEHAVIORAL MODULE

OUTPUT WIRES

Inhibitor

Figure 4.4 . architecture Original AFSM asusedwithin the subsumption

134

Chapter4

....................... .......................................... ................ ........ Back - out - ot . Tight Situations Layer Lost

Collide .. ................................... Clock Explore Layer Go Wander .......

Reverse ...... ............ ..

.. ............. ............ ..... ... .................................. ............ .. S E N S 0 R S ' Run Away : ~ J ~~ S t ~ ForwardS S =~ = = == = ~~ A~ . BRAKES OTORS

Figure 4.5 AFSMs for a simplethree layeredrobot (Brooks 1987b. ) an action and is responsible for its own perception of the world . There is no global memory , bus, or clock . With this design, each behavioral layer can be mapped onto its own processor ( Brooks 1987b) . There are no central world models or global sensor representations. Required sensor inputs are channeled to the consuming behavior. Figure 4.5 shows a simple robot with three behavioral layers . The system was implemented on a radio -control led toy car (Brooks 1987b) . The lowest behavior layer , avoid -objects, either halts or turns away from an obstacle, ' . depending upon the input from the robot s infrared proximity sensors The explore layer permits the robot to move in the absenceof obstacles and cover - - large areas. The highest layer , back out of tight situations , enables the robot to reverse direction in particularly tight quarters where simpler avoidance and exploration behaviors fail to extricate the robot. As can be seen, the initial subsumption language, requiring the specification of low -level A F SMs , was unwieldy for those not thoroughly schooled in its usage. Recognizing this problem , Brooks ( 1990a) developed the Behavior Language, which provides a new abstraction independent of the A F SMs themselves using a single rule set to encode each behavior. This high level language is then compiled to the intermediate AFSM representation, which can then be further compiled to run on a range of target processors.

135

BehaviorBasedArchitectures

4.3 Coordination Subsumption .2 in

The name subsumption arises from the coordination process used between the layered behaviors wi ~ the architecture. Complex actions subsume simpler behaviors. A priority hierarchy fixes the topology . The lower levels in the architecture have no awareness of higher levels. This provides the basis for incremental design. Higher -level competencies are added on top of an already working control system without any modification of those lower levels. The older version of subsumption specified the behavioral layers as collections of A F SMs , whereas the newer version uses behavioral abstractions (in the form of rules) to encapsulate a robot ' s response to incoming sensor data. These abstractions are then compiled into the older AFSM form , but this step is transparent to the developer. Coordination in subsumption has two primary mechanisms: . Inhibition : used to prevent a signal being transmitted along an AFSM wire from reaching the actuators. . Suppression: prevents the current signal from being transmitted and replaces that signal with the suppressing message . Through these mechanisms, priority -based arbitration is enforced. Subsumption permits communication between layers but restricts it heavily . The allowable mechanisms have the following characteristics: . . . . . . low baud rate, no handshaking messagepassing via machine registers output of lower layer accessible for reading by higher level inhibition prevents transmission suppression replaces messagewith suppressing message reset signal restores behavior to original state

The world itself serves as the primary medium of communication . Actions taken by one behavior result in changes within the world and the robot ' s relationship to it . New perceptions of those changescommunicate those results to the other behaviors.

-Based 4.3.3 Design Subsumption in Reactive Systems

The key aspects for design of subsumption- style robots are situatedness and embodiment ( Brooks 1991b) . Situatednessrefers to the robot ' s ability to sense its current surroundings and avoid the use of abstract representations, and em-

136

Chapter4

and the bodimentinsiststhat the robotsbe physicalcreatures thus experience heuristics . world directly ratherthan throughsimulation Mataric 1992apresents of for the designanddevelopment this type of robot for a specifictask. The outlin.edis asfollows: basicprocedure for I . Qualitativelyspecify the behaviorneeded the task that is, describethe , to overall way the robot responds the world. behaviorsas a set of observable and specify the robot' s independent 2. Decompose the disjoint actionsby decomposing qualitativebehaviorspecifiedin step 1. the 3. Determine behavioral ) process granularity(i .e., boundthedecomposition onto sensors actuators and . andgroundthe resultinglow-level behaviors the An additionalguidelineregardingresponse encodingrecommends useof . small motionsratherthan largeballistic onesby resortingto frequentsensing tentativepriorities for , coordinationis imposedby initially establishing Finally . the behaviorsandthen modifying andverifying themexperimentally -style Let us now review the exampleof experimentallydriven subsumption in ) design( Brooks1989a previouslymentioned section3.1.3. The targetrobot is a six-legged walking machinenamedGenghis(figure 3.6). Its high-level is behaviol performance to be capableof walking over rough terrain and to ;'al the . havethe ability to follow a human This constitutes qualitativedescription -level performance in . mentioned step 1 above of task . The next step involving behavioraldecompositionmustnow be performed , , Each of the following behaviorallayers was implemented tested and de, , buggedin turn: : 1. Standup Clearly before the robot can walk, it needsto lift its body off of , of leadsto the development two AFSMs, . the ground Further decomposition to control the leg' s swing position and the other its lift . When all six legs one a behavior the robot assumes stancefrom which it , operateunderthe standup canbegin walking. 2. Simple walk: This requiresthat the leg be lifted off the groundand swung forward (advance. A variety of sensordata is usedto coordinatethe motion ) ' betweenlegs including encoders , returning the position of eachleg s joints. When appropriatelycoordinated a simple walk over smoothterrain (tripod , is achieved . gait) . : 3. Forcebalancing Now the issuesconcerningrough terrain are confronted in to the legs, providing activecomplianceto changes are Forcesensors added ' . the grounds contour

137

Behavior -Based Architectures

4. Leg lifting : This helpswith steppingover obstaclesWhenrequired the leg . , canlift itself muchhigher thannonnal to stepover obstacles . 5. Whiskers Thesesensors addedto anticipatethe presence an obstacle : are of rather than waiting for a collision. This capability emerges important as witlt the previousbehaviors . throughexperiments 6. Pitch stabilization Furtherexperiments : showthat the robot tendsto bump into the groundeitherfore or aft (pitching . An inclinometeris addedto measure ) the robot' s pitch anduseit to compensate preventbumping Now the and . robot' s walking capabilitiesarecomplete . 7. Prowling: The walking robot is now concerned with moving toward a detected human The infrared sensorsare tied in. When no personis present . , . As walking is suppressed soonassomeone stepsin front of the robot, the suppression . stopsandwalking begins 8. Steeredprowling: The final behavior allows the robot to turn toward the personin front of it and follow him. The differencein readingsbetweentwo IR sensors usedto providethe stimulus andthe swingendpointsfor the legs is , on eachsideof the robot aredetermined the differencein strength . by The completedrobot, satisfyingthe task criteria established it, consists for of fifty -sevenAFSMs built in an incrementalmanner Each layer has been . testedexperimentallybefore moving onto the next, and the results of those testshaveestablished needfor additional layers (e.g., whiskersand pitch the stabilization. )

4.3 Foraging .4 Example

-baseddesign The foragingexamplepresented earlier alsoillusb' atessubsumption . In particular the robots Mataric ( 1993a constructedfor severaltasks , ) the basisfor this discussion The robots are programmed . including foraging provide in the Behavior Language The target hardwareis an IS Robotics . system(figure 4.6). Each behaviorin the systemis encodedas a set of rules (standard the for BehaviorLanguage. The overallsystem actuallybeendeveloped amul has as ) tiagentrobotic system(chapter9), but for now we will restrict this discussion to a singlerobot foraging. The following behaviors involved: are . . . . : Wandering movein a randomdirectionfor sometime. : Avoiding turn to the right if the obstacleis on the left, then go. turn to the left if the obstacleis on the right, thengo.

) . MA

Somerville

courtesy , Robotics IS of

photograph ( .

foraging RI : robot based -

Subsumption

Figure 6 . 4

Chapter 4 138

139

BehaviorBasedArchitectures

Figure 4.7

SR diagram for subsumption -based foraging robot .

. after threeattempts backup andturn. , . if an obstacle presenton both sides randomlyturn andback up. is , . Pickup: Turn towardthe sensed attractorand go forward. If at the attractor , closegripper . . Homing: Turn towardthe homebaseand go forward, otherwiseif at home , . stop . Figure4.7 illustratesthe SR diagramfor this setof behaviorsPriority-based arbitrationis the coordinationmechanismandthe robot is executingonly one , behavioralrule at any time. Note in particularthat when the robot senses the attractor wanderingis suppressed when the attractoris grabbed homing and , , then suppress pickup (allowing the robot to ignore the potentialdistraction es of otherattractors might encounter it alongthe way).
4. 3.5 Evaluation

When the criteria presented section4.1.3, are appliedto evaluatethe subsumption in architecturethey identify the following strengths : , . Hardware retargetability Subsumptioncan compile down directly onto : -arraylogic circuitry (Brooks 1987b. ) programmable . Supportfor parallelism Each behaviorallayer can run independentlyand : . asynchronously . Niche targetability Custom behaviorscan be created specific task: for environment pairs.

Thefollowingcharacteristics as nor : emerge neither strength weaknesses

140

Chapter4

. RobustnessThis can be success into thesesystems is but : fully engineered . often hard wired ( Ferrell 1994 andhencehardto implement ) . Timelinessfor developmentSomesupporttools exist for thesesystemsbut : , with custombehavioraldesign . curve is still associated a significantlearning -and error developmentcan slow development , , Experimentaldesigri involving trial with Brooks' philosophy simulatorsare not usedto . Also, consistent , . pretestbehavioralefficiency : Under the criteria, the following showup asweaknesses . Run time flexibility : The priority-based coordinationmechanismthe ad hoc , 's limit flavor of behaviorgeneration and the architecture hard wired aspects , . the waysthe systemcanbe adapted during execution . Supportfor modularity: Although behavioralreuseis possiblethroughthe robots Subsumption . in BehaviorLanguageit is not widely evidenced constructed , on the basisthat sinceupperlayersinterferewith hasalsobeencriticized lower ones they cannotbe designedcompletelyindependently Hartleyand ( , 1991 . Also behaviorscannotalwaysbe prioritized (nor shouldthey ) Pipitone be), leadingto artificial arbitrationschemesHartley andPipitone1991 . Commitment ) ( . is asthe solecoordinationmechanism restrictive to subsumption
4. 3.6 Subsumption Robots

using the subsumption Many different robots (figure 4.8) havebeenconstructed them. They include . architectureBrooks 1990breviewsmanyof -basedrobot, which used sonarfor navigation . Allen: the first subsumption . basedon the ideasin Brooks 1986 . Tom andJerry: two small toy carsequipped with infraredproximity sensors ) (Brooks 1990b. . Genghisand Attila : six-leggedhexapods walking capableof autonomous Brooks 1989a. ) ( . Squirt: a two-ouncerobot that responds light (Flynn et al. 1989 . to ) -based . Toto: the first map constructingsubsumption robot andthe first to use , the BehaviorLanguage Mataric1992b. ) ( . Seymour a visual motion-trackingrobot ( BrooksandFlynn 1989 . : ) . Tito: a robot with stereonavigationalcapabilities(Sarachik1989 . ) . Polly: a robotic tour guidefor the MIT AI lab (HorswiIl1993b). . Cog: a robot modeledas a humanoidfrom the waist up, and used to test theoriesof robot-human interaction and computervision ( Brooksand Stein 1994 . )

141

Behavior - Based Architectures lA -robot

A- eye
Herbert

Tj

ien

courte ) of RodneyBrooks.) S ( photograph

4.4 MOTORSCHEMAS
Another approach more strongly motivatedby the biological sciences appeared , , on the heels of the subsumptionarchitecture This behaviorbased . methodusedschema , theory which we reviewedin chapter2. We recall from that review that schema theoryprovidesthe following capabilitiesforspecifying and designingbehaviorbasedsystems from Arbib 1992 : (adapted ) . Schema control of theoryexplainsmotor behaviorin termsof the concurrent . manydifferent activities . A schema storesboth how to reactandthe way that reactioncanbe realized . . Schema is a distributedmodelof computation . theory . Schema for . theoryprovidesa language connectingactionandperception . Activation levelsare associated with schemas determinetheir readiness that or applicability for acting.

Clint

A - ear

Robot

Climbing Photovore motor Jim Frame 200 Attila Squirt Robot Skiing Jerry and Tom Genghis

142

Chapter4

. Schema theory providesa theory of learningthroughboth schema acquisition and schema tuning. . Schematheory is useful for explaining the brain' s functioning as well as robotics . disbibutedAI applications suchasbehaviorbased ( ) in Schema theory is an attemptto accountfor the commonalities both neuro .Arkin choseit as a suitablevehicle to , biological and artificial behavior and robotic behavior . implement the Arkin (1989a 1990a 1993 addressed implicationsof schema , , ) theoryfor robotics : autonomous 1. Schemas , providelargegrain modularity in contrastto neuralnetworkmodels . motor control andperception the between for expressing relationships , in a cooperative 2. Schemas concurrentlyas individual disbibutedagents act onto disbibutedprocessing yet competingmannerandthusarereadily mappable architectures . 3. Schemas provide a set of behavioralprimitives by which more complex . behaviors(assemblages be constructed ) can of 4. Cognitive and neuroscientificsupportexists for the underpinnings this . Thesecan be modified if appropriate as additionalneuroscientific , , approach . or cognitivemodelsbecomeavailable -based roboticsis to providebehavioralprimitives The overallmethodof schema that can act in a disbibuted parallel mannerto yield intelligent robotic , stimuli. Lyonshasalsousedschema to actionin response environmental theory in his research sections3.2.4.1 and 6.6.3), but herewe focus on the method ( . ology Arkin adopted es methoddiffers from otherbehavioralapproach in several The motor schema : significantways . Behavioralresponses all represented a singleuniform format: vectors in are ) ( encoding. response using a potentialfields approach a continuous generated . Coordinationis achieved means vectoraddition. by throughcooperative . No predefinedhierarchyexists for coordination instead the behaviorsare , ; at run-time basedon the robot' s intentions capabilities and environmental , , configured at or . can constraints Schemas be instantiated deinstantiated any events hencethe structureis more of a dynamically time basedon perceptual , . changingnetworkthan a layeredarchitecture . Purearbitrationis not used instead eachbehaviorcanconbibutein varying , ; ' of . to degrees the robot s overallresponseThe relativestrengths the behaviors ' . the (G) determine robot s overall response

143

-Based Behavior Architecture ~

Table4.2 Motorschema .~
Name Background Precursors Principal designmethod Developer Response encoding Coordination method Programming method Robots fielded References Motor Schemas Reactive of component AuRA Architecture Arbib 1981 Khatib 1985 ; Ethologicallyguided RonaldArkin (GeorgiaTech ) Continuous usingpotentialfield analog via and Cooperative vectorsummation normalization Parameterized behaviorallibraries MARV George Ren and Stimpy Buzz, blizzards , , , , mobile manipulator others , Arkin 1987a Arkin 1989b Arkin 1992a ; ;

' . Perceptual uncertaintycanbe reflectedin the behaviors response allowing by it to serveasan input within the behavioralcomputation . Table4.2 summarizes importantaspects this architectureThe remainder the of . of this sectionstudiesthe detailsof its implementation .
4.4.1 Schema -Based Behaviors

Motor schema behaviorsarerelatively largegrain abstractions reusable over a wide rangeof circumstances . Many of the behaviorshaveinternal parameters that provide additional flexibility in their deployment The behaviorsgenerally . are analogous animal behaviors(section2.4), at leastthoseuseful for to . navigationaltasks A perceptual schema embedded is within eachmotor schemaTheseperceptual . schemas the environmental informationspecificfor that particular provide behavior Perception conducted a needto-know basis individual perceptual . is on : the information necessary a particularbehaviorto for algorithmsprovide react Chapter7 details this sensingparadigm referredto as action-oriented . , . is perception Suffice it to say for now that attachedto eachmotor schema a of providing suitablestimuli, if present asrapidly , perceptual process capable aspossible Perceptual . schemas recursivelydefined thatis, perceptual are subschemas , can extractpiecesof informationthat are subsequently processed by another schema a morebehaviorallymeaningfulunit. An example into perceptual : might involve recognitionof a personwith more than one sensor Infrared

144

Chapter4

~ MAS MOTORSCH
~NVIRONM AL ~NT

SENSORS

E N V I R 0 N M E N T

ROBOT

MOTOR
. . .

. . .
Figure4.9 -action . schema relationships Perception

Key : Schema PS- Perceptual PSS Perceptual SUbschema MS- Motor Schema ES- EnvIronmental Sensor

sensors provide a heatsignature whereas can , computervision may provide a . from eachof theselower-level perceptual humanshape The infonnation generated es process would be mergedinto a higher level interpretationbefore . it on to the motor behavior This enablesthe use of multiple sensors passing within the contextof a singlesensorimotor behavior Figure4.9 illustratesthis . . relationship Each motor schema as output an action vector (consistingof both orientation has and magnitudecomponents that definesthe way the robot should ) move in response the perceivedstimuli. This approachhas beenusedfor to ), ( navigationwhereeachvectoris two dimensional Arkin 1989b ground based navigation for generating three dimensionalvectorsfor usein flying or underwater with manyadditional (Arkin 1992a andfor usein mobile manipulators ), et of freedom(Cameron ale1993 . ) degrees

145

-Based Behavior Architectures havebeendefined including Many different motor schemas , . Move-ahead movein a particularcompass : direction. . Move-to-goal: move towardsa detectedgoal object Two versionsexist of . this schemaballistic andcontrolled. : . Avoid-static obstacle move away from passiveor : nonthreatening navigational barriers . . Dodge sidestep approaching : an ballistic projectile. . Escapemoveawayfrom the projectedinterceptpoint between robot and : the an approaching . predator . Stay on-path: movetowardthe centerof a path, road or . , hallway For three dimensional the , . navigation this becomes stay in-channelschema . Noise: movein a randomdirectionfor a certainamountof time. " Follow-the-leader move to a particular location displacedsomewhat : from a possibly moving object. ( Therobot acts as if it is leashedinvisibly to the moving object.) . Probe movetowardopenareas : . . Dock: approach object from a particulardirection. an . Avoid-past moveawayfrom areas : recentlyvisited. . Move-up, move down, maintain altitude: moveupwardor downwardor fol Iowan isocontourin rough terrain. . Teleautonomyallows humanoperatorto provideinternalbiasto the control : at the samelevel asanotherschema . system Theseare the basic building blocks for autonomous navigationwithin this architecture Figure 4.10 depicts severalof the schemasRememberthat although . . the entirefield is illustrated only a singlevectorneeds be computed to , at the robot' s currentlocation. This ensures . extremelyfast computation The actual encodingsfor severalschemas below where Vmagnitude , appear denotes magnitude theresultantresponse the of vectorand Vdirection represents its orientation : . move to-goal (ballistic): = fixed gain value Vmagnitude . = Vdirection towardsperceived goal. . avoid-static obstacle : 0 for d > S = Vmagnitude ~ * G for R < d ~ S , ford ~ R { 00

146

Chapter4

. . . . . " . . ' . . . . . . . . . . . . . . . . . , . . . . . . . : " . . . ~ p . . . . " ' , . " " : ' : ' : ' : . " " " " " " " " ' ' , " " . . " " " , " \ . . . . " " \ \ , " . . . . , " . .

. . . . , , , " " \

. .

. . . . . t ttt tt tt .

. . . . t "

. . , " " . . .

. . . , " " " , " , , , "

. .

. . . . " , , " , .. , , . . . . . . , .. , , . . , . . , ~ : ~ ~ ~ ~ . . . . . . ' ~ ~ ~ ~ . . . . ' . . , . ' : . . . . . . . . . ' . . . . . . . . . . , . . .

. ,

t t

" " " " t " '

tt \ \ , tt t

.. ..

" ..

"

~ ~ " ' ' " t ~~ : : : : : \ 1. . . . ~ .. ~ .. ... . . ... . . . . .. .. ~ " fi , .. . . . . ... ~ ~ ~ ... ... . . .. ~ , - .. . . - ~ ~ ... . . . . .. ...;,~ ~ - .. . . - ~ ~~~ ~ . . . . .. ~ ~ ~ ~ ~ .. . . . . . . . . . . . . .. . ~ .. . . ~ ~ . . . . .. ~ ~ ~ ~ . ... .. .. .. . . . . ~ ~ ~ ~ ~ . . . . . *' " k" " ii . . . ... . . .. ~ ~~ P . " " "" " ... ... ~ .. ~ ~ .. .. . . ... ... ... .. .. . " " ~ ~ ~ ~ ~ ~ ~ Prrr ~ ~ ~ . ~ .. ~ , . r ~ ~ ~ " ~ , ~ ~ ~ .. , J S ~ ~ ~ \ . . r ~ " " .. ~ ~ . ' JJ ~ ~ ~ \ " . . " " .. .. .. .. i " \ \ ~ S ~ \ " ' . " . " .. .. .. li \ \ ~ J ~ \ " . . ~ ' " .. .. li4 \ ~ ~ ~ \ " . . ~ ' \ . .. II ~ J ~ ~ \ " . " " " " ~ ~ ~ ~ \ IJ " " " I " ~ ~ \ ' " I " " ' II " " " " . . , . . . , , . . , , I . . . . . . , . . , . . . . . . , . . . . . . . . . . . . . . . . . . . . . . . , .. .. .. , .. ,

(A )

Figure 4.10 : schemas(A ) avoid static obstacle ( B) move ahead(toward east , (C) , ) Representative , ) move to- goal, guarded(compareto ballistic versionin figure 3.16 (B , ( D) noise ( E . ) stay on-path, and (F dodge where S = sphere of influence (radial extent of force from the center of the obstacle), R = radius of obstacle, G = gain , and d = distance of robot to center of obstacle. = Vdirection radially along a line from robot to center of obstacle, directed away from the obstacle.

147

-Based Behavior Architectures

~ _ .-

~ . -' - - -

- . - - - - - - - - - -

) Figure 4.10 (continued . stay-an-path

P = Vmagnitude { ~ where

ford > ( W/ 2) * G for d ::5( W/ 2) ,

W = width of path, P = off-path gain, G = on-path gain, and of d = distance robot to centerof path. = . Vdirection alonga line from robot to centerof path headingtowardcenterline ,

148

Chapter4

(C) 4.10 ) Figure (continued

. move- ahead = Vmagnitude fixed gain value. = Vdirection specified compass direction . . noise = Vmagnitude fixed gain value.

Vdirect = randomdirectionchanged ion ). everyp time steps(p denotes persistence It canbe seenthat the actualresponse are computations very simpleandfast.

149

Behavior - Based Architectures

) Figure 4.10 (continued 4.4.2

Schema -Based Coordination

The next issueis how coordinationis accomplished with motor schemasThe . answeris straightforward vector summation All active behaviorscontribute : . to somedegree the robot' s global motion. G , the gain vector determines to the , relativecompositionfor eachbehavior The notion of schema . gainsis loosely . alignedwith the conceptof activationlevelsmentioned previously In a system whereno learningor adaptation permitted, the gain levels remainconstant is throughoutthe run. We will seein chapter8 that this can be modified during execution permit learning Action-selectiontechniquessoonto be described to . , in moredetail, alsoreflect the notion of activationlevels .

~-" -"'rJ '\ ",-'1A "t-,J '."J ""-A \"..t. \.,, I_ ,~ t1 .rJ.~ ,,A .tA ,"1 J1,,J~," ~ ~ -..t;" tl t;''\~ J'\1 1". \, J _'\; A~ ;,JJ r't-A r_1 \.~..lr;\,J .,'~ ,;J 'A ,~." ,-r;'. ~ "\".Ir 't~" ~t~ -J. ;t~ r-r JrJ Jt\ tl1 tl1 \ 1 -t" t ;t~ .-~ ,1 ~ \.. _ -;1 J ;, r\ "' ') . 1 ~ \ ( D

150

Chapter4

) Figure 4.10 (continued

, Returningto coordination eachschemaoutput vector is multiplied by its . associated gain value and addedto all other output vectors The result is a . single global vector That vector must be normalizedin somemanner(often on ) merelyby clipping the magnitude to ensurethat it is executable the robot. . The resultingnormalizedglobal vectoris sentto the robot for execution This -react processis repeatedas rapidly as possible The schemas operate . can sense , asynchronouslyeachdeliveringits dataas quickly as it can. Perceptual sincethe simpleanddistributed generallylimits overallprocessing speed performance is extremelyrapid. Somecare must be motor response computation that is on takento ensure thereactionproduced based informationstill relevant ' datais not too old). Normally, the perceptualalgorithms (i.e., the perceptual

BehaviorBasedArchitec~

~ PROJ ,~ ~ LOCA '"~ COLL -t4t ~ ,~-," ::~. ",". .'-~..",.,'~ ",-.'." .",.,, ., ".,..,. ".."'~ ., ...,~ ,"., .,.,". .,.~ .,,,", .,,. . ." .,,.." . ."..." , ... ..". .... . 4 '" ROBOT () F
) Figure 4.10 (continued action -oriented design ensuresrelatively prompt processing independent of the data source. Figure 4.11 shows several different types of robot paths resulting from these methods. It is interesting to observe some of the biological parallels for these type of systems (figure 4.12) . Section 3.3.2 described certain problems endemic to the use of potential fields , in particular local minima and cyclic behavior. Schema-based systems are not immune to these problems , nor have they been ignored . One of the simplest methods to addressthese problems is through noise, injecting randomness into the behavioral system through the noise schema. Noise is a common technique used to deal with local minima in many gradient descent methods, for

152

4 Chapter

(A)

(B) 4.11 Figure

through . obstacles

object

target to

moving

manipulator mobile

) C

( and

docking ; obstacles static avoid and dimensional three

; schemas

, noise and obstacle

, static avoid 3

goal to move

path on -

stay

) A

( :

paths robot simulated

Representative

( continued

Figure 11 . 4

) C view roan Front

Top view roan

153

Behavior -Based Architectures

Chapter4

(B) Figure4.12

155

BehaviorBasedArchitectures

, example simulatedannealing(Hinton and Sejnowski1986 and mutation in ) -basedcontrol, (Goldberg1989 to namea few. In schema ), geneticalgorithms randomness works in several : in somecases prevents entry into local it the ways " minima, acting as a sort of " reactivegrease (figure 4.13 . In general it is , ) ' -basedsystemto alwaysusefulto inject a small amountof noiseinto a schema . help ensureprogress Balch and Arkin ( 1993 developed anotherschema avoid past to ensure ) , , is madeevenif the robot tendsto stall. Avoid-pastusesa short term progress as representation it retains a timewindow into the past indicating where the robot has been recently It is still reactive however since no path planning . , , for the robot is ever conducted and the output of this behavioris a vector , of the sameform as all the other behaviorsand is combined in the same from recently visited areasthat prevent way. Repulsiveforces are generated the robot from stalling when not at its goal. This approachhas proven very effectivein evendegenerate cases figure 4.14 . ( ) 8 will discussseveral and Additionally chapter adaptive learningtechniques that havebeenappliedto improvenavigationalperformance .

4.4.3 Desian Motor Sehema- Ba.~ in follows :

Systems

The design process for building a schema based robotic system is typically as 1. Characterize the problem domain in terms of the motor behaviors necessary

to accomplish task. the 2. Decompose motor behaviors their mostprimitive level, usingbiological the to studieswhenever feasiblefor guidelines . 3. Developformulasto express robot' s reactionto perceived the environmental events . ' 4. Conductsimplesimulationstudiesassessing desiredbehaviors approximate the in environment . perfonnance the proposed 5. Determinethe perceptual needed satisfythe inputsfor each to requirements motor schema .
) Figure 4.12 (continued : Biological parallels (A ) A schoolof anchovies being attackedby diving birds. ( photograph . courtesyof Gary Bell) . ( B) A herd of sheepin flight from their handlers . ( photograph , courtesyof TempleGrandin ColoradoStateUniversity)

156

Chapter4

(A)

Figure 4.13 -based Effect of noiseon schema :of navigation(A ) Exactcounterbalancing forcescauses . robot to stall; (B) With noiseadded the problemis prevented ,

6. Design specific perceptualalgorithms that extract the required data for eachbehavior utilizing action orientedperception expectationsand focus , , , to of-attentiontechniques ensure efficiency(chapter7). computational 7. Integratethe resultingcontrol systemonto the targetrobot. ' . 8. Testandevaluate systems performance the . 9. Iterateandexpandbehavioralrepertoireasnecessary

.._'\f ..',~ "-~ .,_ \~ ~ ~ ,.'_ ,-.t J "'"1 .-J -_'1 -~, ,\', -,'~ -.+ -~" -~ _? .~ .""t . ", "" ~ .

Behavioral software reusegreatly simplifies this process becauseschemas , developedusing biological guidelinesoften have extensiveutility in many different circumstances .

157

4.13 ) Figure (continued 4.4.4 Foraging Example

Motor schemas have been used in severalimplementations foraging systems of . The first example mirroring the FSA shownin figure 4.2, consistsof , threeassemblages , eachconsistingof up to four behaviors Arkin 1992b. The ( ) primitive behaviorsare . Avoid-static obstacleinstantiated : obstacles differently for eitherenvironmental or otherrobots . . Move-to-goal: changesits attention from the attractor to the home base depending upon the statein which the robot finds itself.

~~ ~\~ ~ "~+ ,~.t 1 I'I ."'~ .? "',_ ,A -,.o J ," . .+,? '\-.\ -1 '" '\Ir \ , ,~J ._I ""t. -'I.A -o, +~ -. ~ \f ~ _" .,_ .,A ';+"._ ~+ .-;."t .,I;0 ,1 _~ A -?;', -'-r~ ~~ . , \\ ~o .l,01 .? + '_ "~ -" 0 o ~ 1 ~ IA . B ) (
Behavior .-BasedArchitectures

158

Chapter4
09 . 0 cGoal Distance Goal To 3 67 . Sensible 0 Gain Motion 00 . 4 20 . 2 Object 00 . Gain 0 Obstacles 66 Past . 33 441 . 20 Gain 0 . Dist 0 0 Noise Direction 80 Contacts . 2 0 -

(A ) Figure 4.14 forcesaregenerated local minima. Repulsive for Avoid-past schema usage overcoming at recently visited areas preventingthe robot from stalling: (A ) without and (B) with , avoid-past behavior (C) deepbox with avoid-past; (0 ) mazewith avoid-past. ;

Persistence 725

Noise Malnitude Steps

( continued

Figure 14 . 4

Goal

053

0 -

Direction

80

Maenitude

803

Goal

To

80

Motion

00

Obstacles

75

Dist

Contacts

140

Steps

00

00

20

Distance -

Sensible

Gain -

lect .

Ob

Gain -

Past

Gain -

Noise

Persistence -

Noise

159

BehaviorBasedArchitectQres

~ ' G ~

Chapter4

C '

II .

0 U

II oIJ 9 004

II . 00

004

II U

004

. ~

1 oIJ

II

'

C 00 0 0040

. ~

11 oIJ

. Q

. 0040

1 ~

110

004

0 Z

II

II

oIJ

004N

II Q

II

. 004

0 Z

) Figure 4.14 (continued

-~

Behavior -Based Architectu .reS

) Figure 4.14 (continued

. Noise: Initially setat a high gainto ensure broadexplorationof the area then , reducedgreatlyuponencountering attractor an . . Figure4.15 depictsthe behavioralconfigurationusing an SR diagram , ) SubsequentlyBalch et at. ( 1995 createdmore complex foraging robots . (figure 4.16 using a similar methodologyfor use in a robot competition ) 9 discuss this multirobot implementation moredetail. es in Chapter
4.4. 5 Evaluation When evaluated using the criteria presented in section 4.1, motor schema based robotic systems are found to have the following strengths:

D (

" ~

. . " ' " " "

" ' .

"

"

"

"

"

"

"

"

"

"

"

"

'

, ~

~ . . . . . . . . . . . . . . , " ' " " " " " " "

, . . . . . . . . . . . . . .

"

" . . . . . . . . . . . . . .

" " " " " " "

"

" " " " " " . . . . . . . . . . . . . .

- -

' .

_ /

' .

. . . . . . . . . . ~ , . . e . C ; ' " ~ . . . , " " " " " , ~ ~ . . . . . . " ' . . . ~

Jot

lIC

. . . . . . . . . . ~

'

"

"

" "

"

"

"

"

"

"

"

"

"

,g~-/ ~ /le ~-t 1101l ~ a' 'etece CIs ., a"-Obot 'eT ~ -t '
/6 ,2

/:'O ""GE ClJ ter -- - ~ 4

163

BehaviorBasedArchitectures

Figure 4.16 -basedrobot. Callisto, a foraging schema

. Supportfor parallelism Schema : theory is a distributedtheory of computation es. are involving multiple parallel process Motor schemas naturally paral lelizable. . Run time flexibility : As schemas software agents instantiatedat run are , time as process and are not hard wired into the architecture it is simple to es , reconfigurethe behavioralcontrol systemat any time. . Timelinessfor development and supportfor modularity: Schemas essentially are softwareobjectsand are by definition modular They can be stored . in behaviorallibraries andare easily reused( MackenzieCameron and Arkin , , 1995 . ) The following is found to be neithera strengthnor a weakness : . RobustnessAs with any reactive system schemascan well cope with : , . changein the environment One deficiency lies in the use of potential field . , analogsfor behavioralcombination which hasseveralwell-known problems methodshoweversuchastheintroductionof noiseandthe avoid-past , , Specific behavior havebeendeveloped circumventthis difficulty. to , The following weaknesses identifiedunderthe evaluationcriteria: are

164

Chapter4

Figure 4.17 -based robots First row (left to right): 10 Callisto and . Several motor schema , , . . , , Ganymede Secondrow: Shannonand Sally. Third row: George Ren andStimpy : . Rear GT Hummer . Niche targetability : Although it is feasible to design niche robots , the generic modular nature of the primitive schemas somewhat discourages the design of very narrowly focused components. . Hardware retargetability : Schema-based systems are essentially software architectures mappable onto hardware multiprocessor systems. They do not provide the hardware compilers that either subsumption or Gapps does. Hardware mappings are feasible , however ( Collins , Arkin , and Henshaw 1993), just not as convenient as with some other systems.

-Based Robots 4.4.6 Schema

The incremental development of the library of motor schemas can be traced through a series of fielded mobile robots (figure 4.17) . Schemas from earlier robots were easily reused for newer machines as they became available. . HARV : An early Denning mobile robot , named after the cocktail Harvey Wallbanger, which reflected its early behavior. This early robot was capable of a wide range of complex behaviors including :

165

BehaviorBasedArchitectures

. Exploration a combinationof avoid static obstacleandnoise . : . Hall following : move aheadin the direction of the hall coupledwith avoid -obstacle enabledsafenavigationdown a corridor. static ; . Wall following or " drunkensailor behavior usefulforgoing throughdoorways : " . A move aheadschema pointing at an angleinto the wall coupledwith avoid static obstacleproduceda behaviorwhere the robot followed the wall and then passed throughthe first openingit found. This enabledthe robot to " completenonspecifictaskssuchas go down the hall and enterthe first door " on your right. . Impatientwaiting: occurredwhile the robot waitedfor a door to open It consisted . immediately amountof noiseanda move to-goal behaviortargeted of a small -static obstacleschema . beyonda closeddoor, coupledwith the avoid . The robot would oscillatein its local minima until the door opened Whenthe because the new opening the robot of stimuluswasno longerpresent , obstacle . movedthrough the doorway This behavioris potentially useful for entering elevators amongother things. The behavioris also referredto as the fly -at-awindow , behaviorsincea fly is attractedtowardsthe light yet repelledby the ? glassat the window, and the behaviorhasa noisy component(panic ) that is . that it may not initially sense to the fly in finding openings actuallyhelpful . Indoor and outdoor navigation demonsrated in severalways including t : . , , , variouscombinationsof the stay on-path avoid static obstacle noise move , . ahead andmove to-goal schemas , . George This Denning DRV-3, namedafter a fictitious GeorgiaTech student : docking (Arkin andMurphy , wasthe first robot to exhibit behaviorbased -pastbehaviors BalchandArkin 1990 teleautonomyArkin 1991 andavoid ( ), ( ), 1993 . ) . Ren and Stimpy: A pair of DenningMRV-2 robotsusedfor dodge escape , , . behavioralresearch , forage andmultiagent . Buzz: Usedin the AAAI competitiondescribed section3.4.4 (Arkin et al. in 1993 . ) . 10 Callisto, GanymedeThree studentconstructed small mobile robotsfor : , and ) multiagentresearch winnersin a robot competition(Balch et al. 1995 . + . Mobile manipulator One of the MRV-2s fitted with a CRS robot arm : et ) (Cameron al. 1993 .

ARCIDTECTURES 4.5 OTHER

We now surveya representative samplingof the wide rangeof otherbehavior 's and that basedarchitectures exist, highlighting eachone approach contribution and , . All sharea philosophyof commitmentto sensing action elimination

166

Chapter4 Table 4. 3 Circuit architecture Name Background Precursors Circuit Architecture Early reactivearchitec ~ Brooks 1986 Nilsson 1984 Barber et al. 1984 a ; ; ; Johnson1983 Situatedactivity L. Kaelbling and S. RosenscheinSRI) ( Discrete(rule based ) Hierarchicalmediation(arbitrationwith abstraction ) Rex andGapps Flakey 1991 ; Kaelbling 1986 Kaelbling andRosenschein

Princi~ design method Developers Response encoding Coordination method method Programming Robots fielded References

or reductionof symbolic representational , knowledge and the use of behavioral units astheir primary building blocks. Eachone' s uniqueness arisesfrom its choiceof coordinationmechanismsthe response , , encodingmethodsused the behavioralgranularity andthe designmethodologyemployed . ,

4.5.1 Circuit Architecture

The circuit architecture is a hybridization of the principles of reactivity as typi fied by the subsumption architecture , the abstractions used in RCS ( Barbera et al. 1984) and Shakey ( Nilsson 1984), and the use of logical formalisms (Johnson 1983) . Table 4.3 summarizes this approach. We discussed aspects of this architecture in section 3.2.4.2, in particular the role of logical formalisms and situated automata. One strength this approach provides involves the use of abstraction through the bundling of reactive behaviors into assemblagesand by arbitration to occur within each level of abstraction, that is , what the allowing designers refer to as hierarchical mediation . Another advantage is the use of formal logic as a means for expressing the behaviors, permit ting compilation into hardware and assisting with the verification of the perfonnance of the resulting robotic system ( Rosenscheinand Kaelbling 1987) . The motivations for this architecture , according to its designers, are typical for behavior -based systems in general: modularity , permit ting incremental development; awareness, tightly coupling sensing to action ; and robustness , being able to perform despite unanticipated circumstances or sensor failure .

167
4.5.2

Behavior -Based Architectures

Action -Selection Action - selection is an architectural approach developed by Pattle Maes in the late 1980s. It uses a dynamic mechanism for behavior selection. Instead of employing a predefined priority -based strategy typified by the subsumption approach , individual behaviors (competence modules) have associated activation levels that ultimately provide the basis for run -time arbitration . A competence module resembles a traditional AI robotic operator with preconditions , add lists and delete lists. Additionally , an activation level is associated with the module that ultimately governs its applicability at any particular time by being above some threshold. The activation level for any particular module is affected by the current situation , higher level goals, spreading activation due to previous or . potentially succeeding events in time , or inhibition from conflicting modules. Activation levels also decay over time . The module with the highest activation level is chosen for execution from the set of all modules whose preconditions are satisfied. The selection process is repeated as rapidly as possible as the world ' s circumstances change about the agent. Because there is no predefined layering of behaviors as in subsumption, it is harder in action - selection to predict the agent' s global performance in a dynamic environment , and thus action - selection has a greater emergent quality . Several global parameters are used to tune the control system, all of which are related to the activation levels (e.g ., activation threshold , amount of activation energy injected) . An advantage of this strategy is flexibility and openness as , the system' s responses are not hard- wired . As an agent' s intentions can influence the activation parameters, higher level goals can also induce performance changes (see Norman - Shallice model from chapter 6 ) . The action - selection approach also shares much in philosophy with schema theory (Arbib 1992), especially regarding the use of activation levels for controlling behavioral performance . Its primary perceived limitation is the lack of real implementations on actual robots and thus no evidence exists of how easily the current competence module formats would perform in real world robotic tasks. Table 4.4 summarizes this architecture ' s characteristics.

4.5. 3 Colony Architecture

ThecolonyarchitectureConne111989b a directdescendent the subsumption of ( ) is architecturethat uses simpler coordination strategies(i .e., suppression . only) and permits a more flexible specificationof behavioralrelations The

168

Chapter4 Table 4.4

Action - selection architecture

Name Background Precursors Principal designmethod Developer Response encoding Coordination method Programming method Robots fielded References

Action-selection Dynamiccompetitionsystem ; Minsky 1986 Hillis 1988 Experimental PattleMaes(Mit ) Discrete Arbitration via action selection modules Competence Simulationsonly
Maes 1990 ; Maes 1989

Table 4.5 Colony architecture Name Background Precursors Principal designmethod Developer Response encoding Coordination method Programming method Robots fielded References Colony architecture Descendent subsumption of architecture ; Minsky 1986 Brooks 1986 / Ethologically guidedexperimental JohnConnell (IBM) Discreterule based only Priority-basedarbitrationwith suppression Similar to subsumption Herbert(mobile manipulator, wheelchair )

colony architecturepennits a treelike ordering for behavioralpriority as opposed found in subsumptionA closer . to thetotal orderingof layeredbehaviors to ethology was also developed using modelsderivedfrom animals , relationship networks . suchas the coastalsnail to justify the useof pure suppression to was Thepinnacleof this architecture Herbert a robot designed wanderabout , the corridorsof the MIT laboratoryandretrievesodacansfor recycling using the . vision, ultrasound and infrared proximity sensorsTable4.5 summarizes , 's . architecture characteristics colony

169

Behavior - Based Architectures

Table 4.6 AnimateAgent Architech Ire Name Background Precurson Principal designmethod Developer encoding Response Coordination method Programming method Robots fielded ,References

architecture Animate agent situated RAPbased activitysystem et 1960 Miller, Galanter Pribram ; Georgeff aI. , and 1987 1986AgreandChapman ; Ullman1985 ; Situated activity of R. James ) Firby( University Chicago NATbasedcontinuous ) ( Sequencing RAPlanguage robot Chip(trashcleaning ) ; , Firby19891995FirbyandSlack1995

4.5.4

Animate Agent Architecture

to The animateagentarchitectureaddstwo components the RAPs discussed in section3.1.3: a skills systemand specialpurpose problemsolving modules , responsetypically ) ( pirby 1995 . The skills providecontinuousenvironmental RAPsprovide an NAT encoding(Slack 1990 (section3.3.2), whereas ) usinga . for mechanism bundling skills useful in particularsituations In a assemblage sense RAPs are relatedto FSA statesand can be usedto sequence through , , , a collection of skills over time (Firby and Slack 1995 . Situations however ) . usedto definethe states(or context and provide the overall designbasis are ) 's . this Table4.6 summarizes architecture key points

4.sis DAMN
The Distributed Architecture for Mobile Navigation (DAMN ) boasts a rather provocative name. Developed by Rosenblatt ( 1995), initially at the Hughes AI Center and subsequently at Carnegie- Mellon University , this behavior-based system has a unique coordination mechanism. The behaviors in DAMN , which was initially touted as a fine - grained alternative to the subsumption architecture (Rosenblatt and Payton 1989), are themselves asynchronous processes each . generating outputs as a collection of votes cast over a range of responses The votes for each behavior can be cast in a variety of ways, including differing . statistical distributions over a range of responses The behavioral arbitration

170

Chapter4
Table 4.7 DAMN Architecture Name Background Precurson Principal design method Developer Response encoding Coordination method Programming method Robots fielded References DAMN (Distributed Architecture for Mobile ) Navigation -sty Fine-grainedsubsumption Ie architecture Brooks 1986 Zadeh1973 ; Experimental Julio Rosenblatt (CMU ) Discretevote sets multiple winner-tate-all arbiters Custom DARPA ALV andUGV vehicles Rosenblatt Payton1989 Rosenblatt1995 and ;

method discussed section3.4.3, is a winner tate-all strategyin which the in , with the largestnumberof votesis selected enacbnentTable4.7 for . response its characteristics . highlights Also unique to the DAMN architectureare the multiple parallel arbiters for both speedand turning control. Arbitration for each of these activities occurscompletely independentlyChapter9 revisits the DAMN architecture . ' ' in examiningthe DefenseAdvancedResearch A Project Agency s ( D RPA s) Unmanned GroundVehicleDemo II Program .
4.5.6 Skill Network Architecture

The skill networkarchitecture a behaviorbased is for system developed graphical animationrather than robotics Indeed the use of behaviorbasedtechniques . , within animationis becomingwidespreadPioneeringwork by Craig . 1987 on his Bolds systemprovideda compelling set of visual behaviors ) Reynolds( for flocks of birds and schoolsof fish. Recentwork by Hodgins and thesetechniques modelnot only graphicalcrea to ) Brogan( 1994 hasextended ' tures responses their fictional environments alsotheir dynamics Zeltzer to but . and Johnson( 1991 however developedthe skill network architecturein a , ), moregeneralway, providing for a variety of computingagents sensing : agents that provide information regardingthe environment skill agentsthat encode , behaviors and goal agentsthat monitor whethercertainconditionshavebeen , ' met. A modificationof Maes action selectionmechanism servesas the basis

BehaviorBasedArchitects Table 4.8 Skill networkarchitecture Name Background Precursors Principal designmethod Developer Response encoding Coordination method Programming method Robots fielded References Skill networkarchitecture Behaviorbasedanimationarchitecture Maes 1989 Badier andWebber1991 ; Etho10 cal/ Expenmen Tai gi David Zeltzer (MIT ) Discrete Action-selection Agent libraries Graphicalanimationsonly ZeltzerandJohnson1991

tions to minimize computation time , an essential aspect of computer-generated animation . The general characteristics of the skill network architecture appear in table 4.8.

4.5.7 Other Efforts

Severalotherbehavioralapproach warrantmentioning es . . BART: The BehavioralArchitecturefor RobotTasks wasan early approach , Kahn 1991 that definedtask behaviorsarbitrarily and provided supportfor ( ) . military robotic missions Of note wasits useof a focus of-attentionmanager to provide situationalcontextfor the selectionof relevantbehaviors A BART . wasdeveloped specifyingbehavioraltasks for . language . Autochthonous behaviorsDeveloped GropenandHenderson 1990 this : ( ), by behavioralapproach useslogical impedance controllersas the basisfor specifying behavioralresponseOf particular interestis its focus on graspingand . 's . manipulationtasksas opposedto robot navigation Gropenand Henderson methodalsoreliesfar moreheavily on representational knowledge(3D models sensor ) than typical reactivesystems data . generated by . Andersonand Donath A behavioralapproachstrongly influencedby etho : -pound robot (Andersonand , logical studiesand fielded on Scarecrowa 4OO -based Donath1991 . The system similar in spirit to the schema , ) , approachuses

172

Chapter4

encodingand vector summationfor potential fields methodsfor the response the coordinationmechanism . . Smarty Cat: Definedin the Smarty Agent Language Lim 1994 similar Cat ( ), in flavor to the BehaviorLanguage Brooks 1990a this behaviorspecification ), ( has approach beentestedon a CybermotionK2A robot. . Dynamic Reaction A behaviorbasedsystemcapableof using goal-based : constraintsin dynamicor rapidly changingworlds (Sanborn1988 . This system ) . wastestedin simulationin trafficworld, a driving simulator . ARC (Artificial Reflex Control) : In this model for robot control systems , . actualbiological reflexes serveasa basisfor their designin robots The model for hasparticularrelevance rehabilitativerobotics in which a prostheticdevice , . suchas an artificial handor limb must coordinatewith an activehuman This control modelhasbeenappliedto hand knee and leg controllersfor potential , , usewithin humanassistive , ) technology(BekeyandTomovic 1986 1990 . . Niche Robot Architecture This architecture developedby Miller ( 1995 : , ), draws on the notion of ecological niches as espoused MacFarland(Mcby in Farlandand Bosser1993 and presented chapter2. It focusesmore on the ) issuesof creatingrobots for specifictasksrather than on rigid philosophical . or to commitments specificbehavioralencodings coordinationstrategiesSeveral real world robotsfit into this paradigm including Rocky ill , a prototype , Mars microrover Fuddbot a simplistic vacuumcleaningrobot; and a robotic , ; . wheelchairtaskedwith assistingthe handicapped

ISSUES 4.6 ARCmTECTURA LDESIGN

We can extract a number of common threads from this diversity of architectural approaches as well as some themes driving the development of these systems. . Analysis versus synthesis. This methodological difference relates to the underlying assumptions regarding just what intelligence is. In some instances, intelligence is perceived as something that can be reduced to an atomic unit that when appropriately organized and replicated can yield high - level intelligent action. In other approaches, abstract pieces of intelligent systems, often extracted from or motivated by biological counterparts, can be used to construct the required robotic performance . . Top-down ( knowledge-driven ) versus bottom -up (data-driven ) design. This aspect relates more closely to experimentation and discovery as a design driver versus a formal analysis and characterization of the requisite knowledge a system needs to possess to manifest intelligent robotic performance . These

173

BehaviorBasedArchitectures

" differencesperhapsparallel to a degreethe " scruffy versusneat dichotomy in AI . . Domainrelevance . To versusdomainindependence someextentthis characteristic the view that thereeither is or is not a singleform of intelligence captures " . . Herethe AI parallelis " weakversusstrong methods . Understanding . intelligence versusintelligent machines The fundamental ' . differencehere lies in the designers goals Biological constraintscan be apof plied in the development a robotic architecturein an effort to understand the . natureof animalintelligence This approach the may compromise utility of who follow this path . the resultingmachineintelligence Often robot architects that havean underlyingassumption intelligenceis fundamentally independent . in of the underlyingsubstrate which it is embeddedThis is merely a ~ orking to , asthereis yet no strongevidence supportit (nor to contradictit ). . hypothesis with the moredirect goal of building usefuland are Otherarchitects concerned with sufficientintelligenceto function within the world in productivemachines relateto biological systems . which they are situated Whetherthesemachines . is not their concern Thesecompetinggoalsand methodsresult in a wide rangeof architectural . es throughoutthis chapter Many roboticistsfeel that approach , aswe haveseen in excellentresponsiveness dynamic methods behaviorbased provide although lost in their eliminating the use of representational environments much is , knowledge have consideredhow representational . knowledge Theseresearchers . into thesebehavioralarchitecturesIn in variousforms can be integrated 5, we will encountervariousmethodsthat can introducerepresentational chapter while maintainingmost if , knowledgeinto reactiverobotic architectures architectures . not all , of their desirableproperties In chapter6 we will study hybrid with not only behaviorbasedarchitectures that attemptto supplement . but additional deliberativeplanning capabilities knowledge representational ' can es 8 discuss how learningand adaptation be introducedinto these Chapter area , systemsanothervery importantresearch .

SUMMARY 4.7 CHAPTER

. A wide range of architectural solutions exist under the behavior -based paradigm . . These architectures, in general, share an aversion to the use of representational knowledge , emphasis on a tight coupling between sensing and action , and decomposition into behavioral units.

174

Chapter4

. Thesearchitectures differ in the granularityof behavioral , decompositioncoordination methods used response , , basisfor development , encodingtechnique andothe factors . ;r . Our working definitionis that robotic architecture the disciplinedevoted is to thedesignof highly specificandindividual robotsfrom a collectionof common softwarebuilding blocks. . Robotic architectures similar in the sense are that they are all ' lUring computable but indeeddiffer significantlyin termsof their organizational components andstructure . . Behaviorbasedarchitectures be evaluated termsof their supportfor can in , hardwareretargetability ecologicalniche fitting, modularity support , parallelism of . , robustnessflexibility , ease developmentandperformance , , . The subsumption architecture a layeredarchitecture is that usesarbitration and finite statemachines its basis It hasbeenimplemented as . strategies augmented on many robotic systemsusing rule-basedencodingsand an experimental . designmethodology . Motor schemas a softwareorienteddynamic reactivearchitecturethat are -layeredand cooperative as opposedto competitive. Vectorsserveas is non ( ) the continuousresponse mechanism with summationas the fundamental encoding coordinationstrategySeveral . robotic systems havebeenimplemented , and the architecturehas had significant influencefrom biological considerations . . Circuit architectures for predominantlyuse logical expressions behavioral , use abstractioncoupled with arbitration and typically follow the , encoding situatedactivity designparadigm . . Action-selection architecturesare dynamic rather than fixed competition . , systemsandthey alsousearbitration . The colony architecture is a simplified version of subsumption more , in its implementation . straightforward . The animateagent architectureusesreactiveaction packages RAPs) and ( methods unfold situationalresponses to over time. sequencing . The DAMN architectureprovides voting mechanisms behavioral response for with a winner take all arbitration mechanism the style in , encodings . ofsubsumption . The skill network architecture particularly well suitedfor graphicalanimation is andusesaction selectiontechniques . . Many other behaviorbasedarchitectures also exist, varying at somelevel from the otherarchitectural . systems

175

Behavior - Based Architect Ures

. Design choices for robotic architects involve issues such as whether to use analysis or synthesis, take a top - down or bottom -up design stance, design for specific domains or be more general, and whether to consider the abstract role of intelligence in general or simply be concerned with building smarter machines.

ChapterS Representational Issues for Behavioral Systems

Knowledgeis to embarkon a journey which . . . will alwaysbe incomplete cannotbe , chartedon a map will neverhalt, cannotbe described . , - DouglasR. Hofstadter The only justification for our conceptsand systemsof conceptsis that they serveto the . represent complexof our experiencesbeyondthis they havenot legitimacy ; - Albert Einstein Thereis no knowledgethat is not power .
- Ralph Waldo Emerson Is knowledge knowable ? If not, how do we know this ? - Woody Allen

Chapter Objectives 1. To developworking definitionsfor knowledgeand knowledgeuse . 2. To explorethe qualitiesof knowledgerepresentation . 3. To understand what typesof knowledgemay be representable usewithin for robotic systems . 4. To determine appropriate the role of world and self-knowledgewithin behaviorbased robotic systems . 5. To studyseveral for representational strategies developed usewithin behavior based . systems

178

ChapterS

5.1 REPRESENTATIONAL KNOWLEDGE

A significantcontroversyexists regardingthe appropriaterole of knowledge within robotic systems Oversimplifying this conflict somewhat we can say . , that behaviorbasedroboticistsgenerallyview the use of symbolic representational knowledgeas an impedimentto efficient and effective robotic control othersarguethat strongforms of representational , whereas knowledgeare needed havea robot performat anythingabovethe level of a lower life form. to In this chapter we attemptto defusethis argumentby first providing some , definitions and characteristics knowledgeand knowledgerepresentations of and then showing successfulexampleswhere knowledgerepresentations of variousforms havebeenintroducedinto reactiverobotic systems the behav at iorallevel. We emphasize appropriate of knowledge not merely using the use , ' for knowledges sake Later, in chapter6, we describe . how it is possible knowledge to exploit multiple robotic control paradigms within a singlearchitecture , with differentcomponents the systememployingknowledge of representations in different ways .

5.1.1 What Is Knowledge ?

. Knowledge , much like intelligence , is a word notoriously difficult to define. Information arises from data, and knowledge can be said to emanate from

' information (figure 5.1). Tanimotos ( 1990 p. 111 definition for knowledge , ) " information in seems context organizedso that it , particularly to the point: canbe readily appliedto solving problems perception andlearning" . , , ' Thrbans ( 1992 p. 792) definitionis alsouseful: " Understandingawareness , , , or familiarity acquiredthrough educationor experience The ability to use . information Knowledgeinvolvesusing information intelligently. To do this ." , , effectively knowledgemust be efficiently organized otherwise it becomes burdensome . Of course if we intend to useknowledgeto guiderobotic behavior it must , , somehowbe represented within the robotic system Much of the debateregarding . ' knowledges role within behaviorbasedsystemscenterson how it is represented within the context of the control system Steels( 1995 considers . ) to involve " physicalstructures for example ( knowledgerepresentations electrochemical states which havecorrelationswith aspects the environment of ) " . andthushavea predictivepowerfor the system Although.this definition is broaderthan most it doescapturetwo very importantcharacteristics : ,

179

Issuesf <r BehavioralSystems Representational ;> Volume

~ .

Organization Figure 5.1 Basisof knowledge

musthavesome 1. Environmental con elation. To be useful, a representation with the externalworld. The natureof that relationshipwill serve relationship that for asa definingcharacteristic manyof the knowledgerepresentations we of . considerIn particular the temporaldurability or persistence the represented , knowledge(e.g., short term, long term) and the nature of the correlational . ) mappingitself (e.g., metric, relational will serveasdefiningfactors 2. Predictive power. This predictiveability is centralto the value of knowledge .H , representation thereis no needto predict we canrely entirely on what . is sensedresulting in a purely reactiveapproach However if there is useful , , information beyondthe robot' s sensingcapabilitiesthat is accurate durable , , andreliable, thenit canbe worthwhile in providing knowledgerepresentations that encodethis information to the robot. As we all know, however the future , is notoriouslydifficult to predict in the real world, evenin the bestof circumstances : . Here lies the controversy in the utility of knowledgeat a given time within a given environment sinceit hasan ability not only to predict but also , , or to deceiveshouldthat informationbe inaccurate untimely. -offs regardingsensingversusrepresenting Figure 5.2 capturessomeof the trade . in various task environmentsWheneverthe world changesrapidly,

180

ChapterS

dynamic end uncerl8ln worlds

highly structuredworld.

FigureS .2 Trade offs for knowledgeuse An increase denoted the arrow' s direction. . is by stored knowledge becomes potentially obsolete quickly . On the other hand, continuous sensing is not free , and from a computational perspective it is better to conduct that process as few times as necessary If the world is unlikely to . change, it can be advantageousto retain previously sensedinformation instead of unnecessarily oversampling the environment. A related problem is maintaining an accurate correlation between the robot ' s position within the world and its representational point of view. This is not a trivial problem . When this involves the spatial location of a mobile robot it is referred to as localization . If maps of any form are maintained , the problem of resolving the robot ' s egocentric frame of reference and the representational frame of reference must be addressed In other words , the question that must . be answered is " Where am I ?" Purely reactive systems do not address this ' question at all , for they are concerned only with the robot s immediate sensations . There is no projection into the future , nor any reasoning over past experience. This chapter delves into various approaches for addressing these trade-offs , maximizing robotic utility in differing circumstances. We must keep in mind as we study these methods that they depend strongly on the environment in which the robotic agent resides and not merely on the architectural choice itself .

5.1.2 Characteristics Knowledge of

suchas Hull andSherring , who preceded behaviorists Early psychologists , ton took the stancethat " knowledgeis embedded the structureof the reflex in " units that generate action (Galliste I1980, p. 336). Therewereneitherdiscrete structures translational nor es the knowledge process to complete mappingfrom store to action. In subsequent , knowledge years behaviorists(epitomizedby " B.F. Skinner , eschewed mentalisticterms such as " knowledge or " mental )

181

Issues Systems Representational for Behavioral " , representations which have no place within the behavioristpoint of view and are viewed merely as artifacts derived from complex stimulusresponse . couplings ) Cognitivepsychologsts (e.gNeisser 1976 haverecentlyforwardedcom~ that and evidence theoriesof knowledgerepresentation havedisplaced pelling as are . much of the earlier behavioriststance Knowledgestructures conceived manipulableunits of information involved in variousways with action generation . Cognitivemaps(Gallistel 1990 are often referredto as the meansfor ) and its translationinto action. Neurobi both storageof previousexperience " " what and where centerswithin the brain ological evidencenow exists for , ), (Mishkin, Ungergleiederand Macko 1983 providing a compellingbasisfor . of theimportance localizationandobjectrecognitionandcategorizationThese be as simple ascollectionsof action vectorsthat maintaina spatial mapsmay events correlationwith sensory arriving from the outsideworld andappropriate I1990 ). actionsby the agent( Bizzi, MussaIvaldi, andGiszter 1991 Galliste ; of is often inextricablyboundup with the issues knowledgerepresentation Learning : for example how arethesecognitivemapscreatedand storedin the , until chapter8, but inevitably ? first place Wedefermostof these learningissues . here someaspects seepinto our discussion This perspectiveon psychologyclearly relatesto behaviorbasedrobotic . in systems that we would like our robotsto act intelligently or knowledgeably ' The debateover knowledges role residesnot in whetherit is useful but rather . within a robotic system in how it appears Traditional AI is often distinguishedfrom behaviorbasedsystemsalong front. Let us considera taxonomyof knowledge the knowledgerepresentation 1982 Malcolm and Smithers1990 : ; ) ( representationsDennett . Explicit : symbolic discrete manipulable , , typical representations knowledge of traditionalAI . . Implicit : Knowledge that is non-explicit but reconstructable and can be . madeexplicit throughprocedural usage . Tacit: Knowledgeembedded es within the systemthat existingprocess cannot . reconstruct , Symbolic systemsuse explicit knowledgeas defined above subsymbolic . involve either implicit or tacit knowledgeuse systems mustuseknowledgeto accomplish It canbe saidthat all intelligent systems . their goals To be truly intelligent the knowledgeusagemust be efficient and , roboticscommunityresistthe useof . effective Thosewithin thebehaviorbased of , , difficulty : the"symbol explicit knowledge aswe havenoted because another

182

ChapterS
groundingproblem. Succinctlystated the symbolgroundingproblemrefersto , the difficulty in connectingthe meaning(semanticsof an arbitrary symbolto ) a real world entity or event It is easyto createa symbol with the intention of . somethi . It is difficult , however to attachthe full meaningand representing , ~g implicationsof that real world object or eventto the symbol. The degeneracy is often recursiveor circular. Othersymbolsareusedto definethe symbolthat one is trying to anchor This only compoundsthe problem by leading to a . of symbolsthat still haveno true meaning We, as humans are . proliferation , capableof groundingour symbols(or language andextractingmeaningfrom ) thempartly because our ability to perceiveandmanipulate environment of the . Meaningarisesfrom our interactionswith objectswithin the world and is not intrinsic to the objectsthemselvesFortunatelyrobotics unlike much of AI , . , us a means which our agentcaninteractwith the world, that is, the provides by robot' s sensors actuators and . If we want our robotsto act knowledgeablyvarioustypesof knowledgeare , . Listed below areseveral : necessary possibilities . Spatialworld knowledge anunderstanding the : of and navigable space structure the surrounding robot. . Object knowledge categoriesor instancesof particular : types of things within the world. . Perceptual : the knowledge informationregardinghow to sense environment undervariouscircumstances . . Behavioralknowledge an understanding how to reactin different situations : of . . Ego knowledge limits on the abilities of the robot' s actionswithin the world : (e.g., speed fuel, etc.) and on what the robot itself can perceive(e.g., sensor , models . ) ' . Intentionalknowledge informationregardingthe agents : goalsandintended actionswithin the environmenta plan of action . Spatial world knowledgecan take severalforms: quantitativeor metric, where someabsolutemeasure usedto establishthe robot' s relationshipto is the world; and qualitativeor relational whereinformation aboutthe world is , described relativeterms(e.g., the goal is just to the right of the second in door on the left). Another way in which knowledgecan be characterized in regardto its is : how long it will be useful lWobasicforms of knowledge persistent . durability , and transitory can be distinguished Persistentknowledgeinvolves a priori . , information about the robot' s environmentthat can be consideredrelatively

183

Issues Representational for Behavioral Systems ' staticfor themissions or task' s duration Thesedatatypically arisefrom object . modelsof things the robot might expect to seewithin its world, models of the free spacewhere it moves and an ego model of the robot itself. The , knowledgebase within which this information residesis termed long-term ' . memory( LTM), indicativeof this datas persistence The robot acquirestransitory knowledgedynamically as it movesthrough theworld andstores in short term memory(STM). World modelsconstructed it from sensorydata typically fall into this category Although STM is rarely . useddirectly for reactivecontrol, it can be appliedwhen purely reactivetechniques encounterdifficulties. In behaviorbasedrobotic systemsdynamically , world modelsshouldbe usedonly when the control regimefails to acquired . , cope with difficult situations Even then STM is best usedto reconfigurethe control systemratherthanto supplantit. Transitoryknowledgeis typically forgotten (fades asthe robot movesawayfrom the localewherethat information ) wasgathered . For both persistent transitoryknowledge the choiceof representational and , structureandformat is lessimportantthanmerelythe availability of the knowledge itself. Persistent ideasof knowledgeallows for the use of preconceived the robot' s relationship to the world, enabling more efficient use of its resources than would be accomplished otherwise Either form of knowledge . , if misused could interfere with the simplicity and efficiency of reactivecontrol , . Nonethelesswhendifficulties with a behavioralcontrol regimearise it is , , useful to provide a bigger picture to help resolvethem. This can result in solutions to problemssuchasthe fty-at-the-window situationin reactivecontrol, whenan insectstrivesto go towardsunlightenteringfrom an outsidewindow, is rebuffedby the glassybarrier expendsall of its energytrying to solve the , with its fixed set of behaviors andultimately dies. If transitoryenvironmental , problem modelsareconstructed undertheseconditions(STM), a robot could usethe informationto circumnavigate barrier the . We previouslydiscussed STM andLTM in box 2. STMperforms over an intennediatetime scale in contrastto LTM , which is more durative and reflexive , , which is quasi instantaneous . Strongneuroscientific evidence , memory existsthat STM process are distinct from LTM (Guigonand Burnod 1995 . es ) LTM is persistent is generallyviewedas the basisfor learning Figure 5.3 and . illustratestheserelationships . Associativememory is anotherfonn of knowledgerepresentationoften , linked with neural network models (Anderson 1995 . Here input patterns , ) often only partially complete evokeresponses memoriesencoded a network or in , model. Variousmappingsare found within the nervoussystem Primary .

184

ChapterS

Transitoxv :Knowled ~e
Purely ReacI Jve I Instantaneous

Persistent Knowledi !:e

A Priori Maps .

Sensor -acquired Maps

Short-term Memory

Long term

Memo ~'

Time Horizon
Figure 5.3 Time horizonfor knowledge .

, examplesinclude retinotopic maps spacepreservingmappingsof the retina onto the visual cortex; and somatotopic of , maps topographicrepresentations thebody surface ontothe brain (Florence Kaas1995 . Chapter8 and ) projecting revisitsneuralnetworkmodelsfor behaviorbased robotic systems . The philosophiesregardingknowledgeuse itself form two subdivisionsof " . , metaphysicsAs defined by Webster1984 epistemologyis the study or a theory of the natureand groundsof knowledgeespeciallywith referenceto its limits and validity." Ontology on the otherhand is definedas " a particular , , ." theory aboutthe natureof being or the kinds of existents considerationsin our context concernthe useandvalidity , , Epistemological . of knowledgewithin behaviorbasedrobotic systems Ontological considerations are more specific to the representational choicesand detenninewhat kinds of things exist (or are understandablewithin the framework of our ) of the robotic system In the remainder this chapter we primarily address onto. , : factorsin our examples knowledgeuse what knowledgeis available of logical choices(i.e., vocabulariesmade . to the systemandthe representational ) 5.2 REPRESENTATIONAL KNOWLEDGE FOR BEHAVIOR -BASED SYSTEMS
which is idle. A little knowledge actsis worth infinitely morethanmuchknowledge that - Khalil Gibran

. , Any numberof animalsusecognitivemaps Foraginganimals suchasinsects near andbirds, arebelievedto usepolar vectorrepresentations their homesites 1989 ( Waterman ). Thesevectorsareimposedover a polar grid coordinatesystem . landmarks Gallistel ( 1990 presents createdby a mosaicof recognizable ) evidence theuseof cognitivemapsin animalsystemsincluding for , compelling the following :

185

Issuesfor BehavioralSystems Representational

. Evidencefor vector spaces animalscapableof spatialencodingincludes in tectal mapsfor angulardeviationin birds andauditorycortexmapsforencoding distanceandvelocity in the bat, amongothers . . Many animalsarecapableof distancetriangulationfrom known landmarks , ~ , , , , including many insects suchas locusts wasps and bees and other animals suchasthe gerbil. . Cognitivemapsin naturehavefound usesin foraging homing, puddlejumping , resourcelocation (e.g., refinding calcium depotsfor the desert (gobies ), tortoise avoidingnearlyundetectable obstacles remembering their location ), by (bats orientingtowardshiddengoals(rats swimmingforplatforms), route selection ), basedon relative distance(chameleons remembering ), placespassed en routeto anotherlocation(mazefinding in rats androute selectionbetween ), in ). placesencountered the past(chimpanzees . Of particular interestis the localization of a geometricmodule within the hippo campusof the rat brain, believedcapableof encodingmetric relations and compasssenseand performing the necessary transformations make to the information useful to the rat. Below we discussan exampleof a robotic ' representation systeminspiredby the rat s hippocampus(Mataric 1990 . ) ' useof Of course animals , cognitivemapsis not a sufficientlycompellingargument for including them within robotic systemsAt the very least however . , , it shouldcertainly give the reactiveroboticistpauseto considertheir potential useandimpactuponperformanceIndeed the principle of biological economy . , that systems generallynot includedwithin biological systems are unless argues someutility or advantage the animal. to they provide can Knowledgerepresentations be made availablein severalways to the control system for behaviorbasedrobotic systems Each compromisesthe . reactivephilosophy(chapter3) to varying degrees providesbenefits but purely difficult to realizewithin the strictestview of behaviorbasedrobotics These . es representational approach correlateclosely to our notionsof transitory and in persistent knowledgemanifested variousforms of STM andLTM. . Short term behavioralmemory Isolating representational : knowledgeto a behavor providesknowledgeon a needto-know basis in a manner , specific ~ similar to action orientedperception section4.4.1). Analogouslywe canrefer ( to this useasaction orientedknowledgerepresentation . Only that information for the performance a specificbehavioris represented of . This preserves necessary thebehavioralmodularityandopportunityfor incremental development that is so valuablefrom the behaviorbaseddesignperspective .

186

Chapter5

. Sensorderived long-term cognitive maps Information that is directly perceived : from the environmentand gatheredonly during the lifetime of the robot' s experiencein a particular environmentis used to constructa stand aloneworld model. This model is plastic in the sense that as new sensordata arrivesthe modelis continuouslyupdated modifiedattemptingto maintain and a closecorrelationwith the actualworld. Theserepresentations constructed are in a behaviorindependent mannerandmay provide informationto the overall behavioralcontrol systemin a variety of ways. The greatestdifficulties lie in with maintaininga high fidelity correspondence the world, especiallywhen it is dynamicandunpredictable . . A priori map derived representations : With thesemethods information is , introducedbeyondthe robot' s sensingcapabilities It may be compiled from . ftoorplansor externalmapsof the world, but it was gatheredindependently of the robotic agent The strengthof theserepresentations in their ability . lies to provide expectations the environmentevenbefore the robot has regarding enteredit. Significantproblemsarise from the fact that the initial knowledge sourcemay be inaccurateuntimely, or just plain wrong, and that the form of , the initial representation may not be easily or convenientlytranslatedinto a usefulrobotic format. We now review specific instances behaviorbasedrobotic systemscapable of of taking advantage knowledgein thesevariousforms. of
5.2.1 Short - Term Behavioral Memory . , Knowledgeis soonchangedthen lost in the mist, an echohalf-heard - Gene Wolfe

Behavioralmemoryprovidescertainadvantages a robot: It reduces need to the for frequentsensor , samplingin reason ably stableenvironmentsandit provides recentinformationto guide the robot that is outsideof its sensory . range Representations that fit this particularcategoryhavethreegeneralcharacteristics . First, they are usedin supportof a singlebehaviorin a behavioralcontrol system . , , most often obstacleavoidanceSecond the representation directly feeds the behaviorratherthan directly tying it to a sensor In essencethe memory . , serves a buffer andtranslatorfor a limited numberof previoussensings as (figure 5.4). Third, they are transitory the representations constructed used : are , . while the robot is in the environment and then discarded They must be reconstructed , if the robot reenters environmentAlthough initially this might the .

187

Issues BehavioralSystems for Representational

Figure5.4 Behavioral . memory absentmindedit is actually , appearto penalizethe robot by making it somewhat valuableas it eliminatesmuch of the difficulty associated with long-term . localization(i .e., havingthe robot maintainits bearingsrelativeto a map) and wherethe position of obstacles is well suitedfor somewhat dynamicenvironments over time. In general purely reactivesystemsstill have , may change in an advantage very dynamic worlds (e.g., navigatingalong a crowdedsidewalk can ), but somebehavioralSTM techniques dealquite well evenwith these . situations Both behavioralmemoryandcognitivemappingcommonlyusegrids to represent are the navigable aroundtherobot. Grid representations arbitrarily space the tessellated regionssurrounding robot. Theycanvary in the following ways: . Resolution the amountof areaeachgrid unit covers(e.g., an inch, a meter : , or more . ) . Shape mostfrequentlysquarebut alsoin otherforms suchasradial sectors : , ) (Malkin andAddanki 1990 . . Uniformity : the grid cells may aUbe the samesize, or they may vary. The most commonvariable sizedgrid methodologyinvolvesthe use of quadtrees et ), (Andresen al. 1985 which areformedthroughthe recursivedecomposition of free space . of Figure 5.5 depictsa few variationsof grids. Implementations grid-based -dimensionalarrays but occasionally most often use preallocatedtwo , representations ) (especiallyfor quadtrees involve linked lists as the central data structure . in We discussed section4.4.2 one exampleof behavioralmemoryusedfor . : during stagnation navigation the avoid-pastbehavior This behaviorprevented navigation by adding repulsion from places the robot has recently visited. A regular grid stored sensoryinformation concerningwhere the robot had

188

Chapter5

I~III,' ,. ~IIased II -schema Pasti'1iiiiiii -~ ' Positionallnfon M.apper Grid B .Satial ( erceptual p -) IIII Memo IL
(shaft encoders ) Avoidance Response Figure 5.6 Avoid-past .

Figure 5.5 Grid representationsA ) regulargrid, ( B) sectorgrid, and(C) quadtree :( .

B (~ )-L III L

recently been (figure 5.6). The original work ( Balch and Arkin 1993 used ) deadreckoninginformationbasedon shaftencoderreadingsfor incrementing counter valueswithin the grid. Other more effective sensors could also be , , , , readily used however such as global positioning systems(GPS infrared ), bar code readers or inertial navigationsystems(INS) with no impact on the , behavioror the representation itself (assumingthe sensorcould provide the resolution . The avoid pastbehaviorwasfully self-containedin that ) necessary it wasconstructed with the intent of usinga representation from the onset and , was completelymodularand integrablewith all of the other behaviorswithin the control system . Yamauchi 1990 developed differentforms of behavioralmemory The two . ( ) first, referredto as wall memory usesan array of elements corresponding to , , the numberof ultrasonic sensors to increaseconfidenceover time that the , robot is near a wall. The memory readingsare then usedto supporta wallfollowing behavior(figure 5.7). Yamauchi further extends notion of behavioralmemoryby storingin an the action memoryinformationnot only aboutthe world (e.g., walls) but alsoabout the robot' s most recent responsesThis memory permits the robot to favor .

189

Representational Issues for ~ ehaviora1 Systems

Figure5.7 . Wallmemory

Figure 5.8 " vision avoidance ~ing stereo . Real time obstacle the direction in which it is currently moving by using a weighted average of past responses to bias the immediate reactive response. This tends to remove noise and reduce the possibility of premature reflexive action due to a single reading . Badal et ale ( 1994) developed an example of real-time obstacle avoidance using stereo vision with a form of behavioral memory . An instantaneous obstacle map stores detected obstacle points projected onto the ground plane. To produce the steering behavior for a high mobility multipurpose wheeled vehicle are mapped onto a polar occupancy grid ( sector (I - IMMWV ), these points based) using a coarse configuration space approach, which then generates a steering vector in the direction of the least hindrance (i .e., the direction with the farthest possible avoidance distance) . This behavior is depicted schematically in Figure 5.8. Borenstein and Koren ( 1989) patented a method for conducting obstacle avoidance in real time using sonar data and a grid - based representation. Their work centers on the use of a certainty grid , an outgrowth of earlier work by ' Moravec and Elfes ( 1985) . Moravec s work focused on constructing a world

Response

iiiiiiii

Map

Instantaneous

Obstacle

Detector

Visio

Algorithm

Algorithm

Wall

Sonar

. Stereo Video Cameras Two

Response

190

ChapterS

Sonar Readings

Obstacle Detection

Figure 5.9 Vectorfield histogramusa2e obstacle for avoidance .

- ~~ L ~ -I- D ~ .Gridimen L -One Certainty --~1Polar :1 -~LHistog L L

"

D~ ~ ~

model using sonardata coupledwith probabilistic sensormodelsthat, when . given multiple readingsof the world, could createa grid-basedmap Traditional path planning was then conductedwithin this representational framework to producea route for the robot, which would then move through the world in a nonreactive manner Matthies and Elfes ( 1988 later extended . this ) work to provide sensor fusion of both sonarandvisual stereodata . Borenstein Koren( 1989 modifiedthis methodsothatmanymoresensor and ) couldbetakenby greatlysimplifying the world mapupdatingprocess . readings Informationcouldbe added from the incomingdata Further they . continuously , addeddecayprocess to decrement valuesovertime, ensuringcurrencyof es cell the data Initially the grid took the traditional two-dimensionalsquareformat . in their vector force field concept A repulsivevector generated a manner . in , to that of the potential fields method(section3.3.2), producedthe analogous steeringdirection and velocity for the robot. In later work ( 1991 Borenstein ), andKorenalteredthe representational format to includea one dimensional polar form, the vector field histogram further decreasing processing the time by , collapsingthe two-dimensionalgrid directly into a directional representation centered the robot' s currentlocation. Each sectorstoresthe polar obstacle on . trivial , simply selectingthe most density The steeringcomputationbecomes suitabledirectionin the histogramconsistent with the robot' s overallgoal. Figure 5.9 depictsthis approach . All of the examples discussed thusfar havebeenconcerned with short term datathat is used then discarded and that is channeled , , directly to a particular behavior Wenow look at representational . methods aremorepersistent that and . potentially havebroaderusefor a variety of behavioralresponses

5.2.2 Long-TermMemoryMaps
Under some circumstances, persistent infonnation regarding the environment may be useful for behavior -based robotic systems. In general, these long -term

191

Issuesfor Behavioral Representational Systems

are representations bestusedto advisea behavioralcontrol regimeratherthan dictateto it. The origin of the map data itself provides a useful way to classify these . onboardthe robot, as the robot maps Someare deriveddirectly from sensors movesaboutthe world storinginformationin a particularrepresentational format for later use Othersare constructed . from information gatheredindependently of the robot but transformedinto someuseful format. Thesea priori maps are not as timely as sensorderived maps but can be obtainedfrom a broaderrange of resources even including remote sensingdevicessuch as , satellites . The map representational knowledgeitself is typically encodedin one of two forms: . Metric: in which absolutemeasurements coordinatesystemsare used and to represent informationregardingthe world. Latitudeandlongitudemeasurements aretypical of this format. . Qualitative in which salientfeatures their relationships : and within the world arerepresented . This may supportbehavioraldescriptions suchas " turn left at the seconddoor on the right," or " continue moving until you seethe sign." Thereis little or no notion of any quantitativemeasurement within the world, . only spatialor temporalrelationships The useof anyform of mapknowledge dangerous is primarily in the fact that it may be untimely (i.e., the world haschanged sincethe mapwasconstructed ) and henceinaccurate Additionally, localization a nontrivial perceptualactivity . , ' providing environmentalcorrelation of the robot s position within the . mappedworld, needsto be conducted Map knowledgeis advantageous pri in that mapscan provide guidancebeyondthe horizon of immediate marily . sensingThesetrade offs needto be weighedvery carefully whenconsidering for . roboticistwho mapusage particulartaskenvironmentsThe behaviorbased chooses use mapsstrivesto permit immediatesensoryactivity to override to any potentiallyerroneous map information.

-DerivedCognitive 5.2.2.1 Sensor Maps

Sensor- derived maps provide information directly gleaned from the robot ' s experiences within the world . How long the information is retained will determine its timeliness . As the world is sampled from the robot ' s egocentric point of view , it is often advantageous to use qualitative representations instead of metric ones becauseof the inherent inaccuracies in robot motion and the sensor

192

ChapterS
-

III
E [ =

II

Figure5.10 : Distinctive ; (B) depth regions(A) endof hall threewaysymmetry doorway abrupt constriction - depth minimumD ) visualconstellations C) hallway ;( discontinuity of lines in thiscasetwotriplets vertical ). feature , ( patterns unique . , readingsthemselvesIn behaviorbasedsystems sincethe information is intended the only to supplement reactivecontrol systemrather than to replace it , which is not the casewith traditional robotic motion planning ( Latombe in 1991 inaccuracies the sensordataare toleratedin a much more forgiving ), manner . is One of the hallmarksof qualitativenavigationaltechniques the notion of that distinguish distinctive places regions in the world that have characteristics , them from their surroundings Kuipers and Byun 1988 . A distinctive ) ( " placeis definedas the local maximumfound by a hill -climbing control strategy " measure (Kuipers and Byun 1991 . ) given an appropriatedistinctiveness -level topologicaland geometricrepresentations alwaysderivedin are Higher . observationsIn robotic this systemfrom thesesemantically sensory grounded are thesecharacteristics most often derivedfrom the sensor , readings systems . themselvesLocalesthat exhibit symmetryin somemanner abruptdiscontinu , of ities in sensorreadings unusualconstellations sensorreadings or a point , , where a maximum or minimum sensorreading occursare typical examples and theseobservable ultimately recognizable ) (figure 5.10 . After the robot detennines for lower-level control, such as landmarks they can later be used , . space moving to a particularpoint in this observation As thesemap featuresare often directly tied to sensing integrationinto a , . behavioris fairly easyFor example the robotcanreadilybe instructedto move , then switch aheaduntil an abruptdepthdiscontinuityoccurson the right -and -through doorbehavior The sensorderivedqualitativemapserves as . to a move the basisfor behavioralconfigurationandaction. Of course obviousproblems , to ariseshouldthe doorhappen be closed but that is typical of relianceon map , to of any sort. The world is assumed exist asit is modeled which mayor data , . may not be the case
l - . J

"

193

Representational Issues for ,Behavioral

v ~tern ~

Figure5.11 -style of in . Integration representationsubsumption architecture s' is Qualitativerepresentation primary advantage their relativeimmunity to errorsin motion, especiallywhenthe robot reliesheavily upondeadreckoning as its basisfor localization An examplesystemusedfor outdoor navigation . usesviewframes representations constructedfrom visual input that possess , the qualitiesof distinctiveness discussed earlier (Levitt andLawton 1990 . As ) such they constitutea visual memory of where the robot has beenand provide landmarksregardingits position within the world. Path transformations link aggregations viewframestogetherand describehow the robotic agent of movedfrom oneplaceto the other The resultingnetworkconstitutes . asensor derivedmap useful in instructing the robot in how to navigate This form of . qualitativenavigationcanbe readily tied to behaviorbased ( systemsArkin and Lawton 1990 Lawton, Arkin , andCameron1990 . In particular reactivecontrol ; , ) canprovidea safebasisfor exploringunknownareas during the qualitative . Behaviors be developed arecapable attractingthe can that of mappingprocess robotic agentto areas it hasnot yet surveyedtheycanprovidethe ability to that ; track relativeto qualitativelandmarks permit recoveryfrom disorientation and for events . by allowing the robot to search specificperceptual Mataric ( 1992b demonstratedthe integration of qualitative maps and ) -basedsystem Figure 5.11 behaviorbasedrobotic systemsin a subsumption . -style controller Landmarksare derived from sonar . depicts the subsumption data using featuresthat are stable and consistentover time. In particular , , right walls, left walls, and corridors are the landmarkfeaturesderived from the sensorsSpatialrelationshipsare addedconnectingthe variouslandmarks . a . of throughconstructing graph Map navigationconsists following the boundaries of wall or corridor regions When the robot, Toto, determinesthat it . is in a particular region it can move to different locales by traversingthe ,

J. .

'

~ . , . .

S E N S 0 R S

Response

194

Chapter5

that connectsdifferent regions effectively conducting , graph representation . of , path planning Nonethelessthe advantages reactivenavigation~ not lost, as the robot is not requiredto follow its path blindly throughthe world when other overriding sensordata indicatethat it is more importantto detour(e.g., for obstacleavoidanCe Of additional interest is the claim that this mode ). of navigationalbehavioris a possibleinterpretationof the mannerin which the rat' s brain (specifically the hippocampus conductsnavigation ( Mataric ) 1990 . ) Several othervariationsof qualitativemapssuitablefor behaviorbased navigation havebeendevelopedincluding , . A behaviorbasedcontroller capableof such actionsas hallway following andcomer turning is coupledwith a finite stateautomata of representation the -first search world (Basye 1992 . The navigationaltask then becomes depth a ) ) through the statetransition graph (Dean et al. 1995 . This systemhas been and developed predominantlyfor officelike environments to datehashadonly limited experimental . testing . A modelderivedfrom panoramic visual sensing a 360-degree field of view) ( basedon depth recoveryfrom the motion of .a robot (Ishiguro et al. 1994 . ) not specificallyfocusedon reactiverobotic systems it is potentially , Though . useful in that regard A two-and a-half- dimensionaloutline structureof the environment contourdepthalong a line) is recovered ( using visual depthfrom . motion techniquesand broken into consistentvisual features The resulting is convertedinto a qualitativeforln by segmenting panoramicrepresentation . the outline model into objects basedon abrupt depth discontinuities This visual event prediction along the robot' s qualitative model can be used for . intendednavigationalpath or for localizationpurposes neednot be derivedsolelyfrom sensory . Let us now investigate data Map knowledge of someothermethods mapconstruction potentialuseinbehaviorof . basedrobotics 5.2. 2.2 A Priori Map -Derived Representations A priori maps are constructedfrom data obtained independentlyfrom the for robotic agentitself. The mostcompellingarguments usingthis type of map and : arisefrom convenience greaterscope knowledge . It may be easierto compile thesedatadirectly without forcing the robot to of travel throughthe entire world ahead time.

195

Representational Issues for ,Behavioral Systems

. These data may be available from standard sources such as the Defense Mapping Agency or the U .S. Geographical Survey, among others. . Precompiled sources of information may be used, such as blueprints , floor , and roadmaps, tha~need only to be encoded for the robot ' s use. plans Of course, the perils to the accuracy of this data are different since it comes from other sources: . Errors may be introduced in the process of encoding the new data. . The data may be relatively old compared to recent robotic sensor readings. . The frame of reference for the observations may be somewhat incompatible with the robot ' s point of view . These are just some of the trade-offs that must be considered when using a . priori map knowledge . Clearly , a behavior -based roboticist ' s goal would be to include this information in a manner that does not impede the robot ' s reactive ' performance , but that allows for guidance from knowledge outside the robot s direct experience. Payton ( 1991) provides one of the most compelling examples of use of apri ori map knowledge within the behavior-based robotics paradigm . In Payton' s ' research, fielded in D A RPA s Autonomous Land Vehicle Program, a map of the environment containing known obstacles, terrain information , and a goal location is provided in a grid -based format derived from a digital terrain map. A cost is associated with each grid cell based on mission criteria ; the cost can take into account such factors as traversability , visibility to the enemy, ease of finding landmarks, and impact on fuel consumption , among others. A gradient field is computed over the entire map from start point to goal point with the minimum cost direction represented within each cell to get to the goal . Figure 5.12 depicts an example map. The gradient field represents what is referred to as an internalized plan , since it contains the preferred direction of motion to ' accomplish the mission s goals. The key to this approach' s success is its integration with the other behaviors within the overall control system. The internalized plan acts like another behavior sitting atop a subsumption- style architecture (figure 5.13) . The lower level behaviors guide the vehicle when the situation warrants, but if the vehicle is proceeding normally , the highest-level action that corresponds to the inter nalized plan representing the overall mission is enacted. This method of injecting map knowledge into reactive control is also readily applicable to other behavior -based architectures.

196

Chapter5

.ce-QI.c cn . >(Q .~ 1 0 rCcn ; Q

Figure5.12 from field an Gradient representing internalized . (Reprinted PaytonD., " Internal , plan " for izedPlansA Representation ActionResources1991pp. 94, with kind permis : , , 24, KV , The Burgerhartstraat 1055 , Amsterdam

INTERNALIZED PLANS

Cl

RESPONSE
Figure5.13 internalized . control Behavioral using plans

197

Issues BehavioralSystems for Representational

G 0 A L S S E N S
0 R S
Figure 5.14 Action-selectioncontrol architecture 1995 (Simmonsand Koenig ).

RESPONSE

Simmonsand Koenig ( 1995 presenta different approach encodingboth to ) and metric information from mapsobtainedfrom ftoorplans The . topological baserepresentation constructed is from topological (connectivity modelsof ) theenvironmentcommonknowledge aboutoffice structure(e.g., corridorsare , of , and approximatemeasurement the environmentregardingwidth ) straight of passageways distances and between ) turning (decision points. The resulting the graphrepresents world. Markov models(specialized ) probabilisticmodels encode actionsthat a robot cantakeat different locationswithin the model the . As the layout of the office chosenis simple the allowableactionsare simply , or . turning right or left 90 degrees proceedingstraight aheadfor one meter Obstacle avoidance handledusingthe sameapproach in Arkin 1987 , and is as a with on to supplemented plannedactionsbased wherethe robot needs proceed to its mission The planneritself usesan A * searchalgorithm to . according . specify the actions to be taken at each point within the topological model The Markov model based directivesto the robot for turning left plannerissues or right, moving forward or stopping The navigationalarchitectureusesan . action selectionmechanism (figure 5.14 . The specificarbitrationmechanism ) usedis a best actionstrategybased the highestprobability for eachpossible on , directive The probability for a directivedepends the probabilitiesderived . on from sensing assess the locationof the robotwithin the map. This system that es hasbeenfieldedon a mobile robot namedXAVIER (built on a RWI B24 base ) 88 percentof its (figure 5.15 and is reportedto havesuccess ) fully completed missionsusingthis strategyin morethana kilometerof total distance traveled .

198

ChapterS

Figure5.15 . XAVIER (photograph , , courtesyof Reid Simmons The RoboticsInstitute Carnegie .) MellonUniversity

199

RepresentationaJ Issues .BehavioralSystems for

Figure 5.16 NavlabII . (Photograph courtesyof The RoboticsInstitute, CarnegieMellon .) University

Another system capable of integrating a priori map knowledge into a behaviorbasedcontrol system was fielded on the Navlab II robot testbed Stentz and Hebert 1995 (figure 5.16 . An eight-connectedcartesiangrid( ) ) basedrepresentation , , capableof storing complete partially complete or no knowledgeof the world is availableto a DynamicA * or D* planner(Stentz 1994 . D* hasthe specialability to replan very efficiently shouldsensordata ) . updatethe storedmap representationThis in essence providesa variant on ' s work discussed earlier that can provide rapid updatesto the map , , Payton basedon incoming sensorydata In the overall system an obstacleavoidance . , behavior basedon range data provides local navigation abilities. The DAMN steeringarbiter (section4.5.5) chooses correct behavioralaction the

200

Chapter5

Goal Seeking
Cell Grid Updates ~ SENSORS

ObstacleAvoidance

5.17 Figure . control Navigational system D* asabehavior using

for the circumstancesfigure 5.17 . The systemwastestedat a slagheapnear ( ) of success in excess 1. kilometersin a clutteredenvironment , fully Pittsburgh driving on its way to the goal. The authorsclaim it to be the first systemthat exhibits on a real robot both efficient goal acquisitionand obstacleavoidance . in an unstructured outdoorenvironment Anothertype of map, thepurposivemap (Zelinsky et aI. 1995 Zelinsky and ; 1996 storesinformation regardingthe utility of behaviorsduring ), Kuniyoshi navigationwithin the world (as opposedto spatialinformation regardingthe ' world' s structures. It is similar in spirit to Yamauchis action memory described ) ' s research the behavioralcontroller consistsof a . earlier In Zelinsky , -like arbiter managingbehaviorssuch as wander collide, wall, , subsumption -an-opening move to-goal, andthe like. The purposivemapmonitors follow, find , behavioralstate The map itself lists featuresand associated . and coordinates . betweenfeatures scalarquantitiesthat estimatethe spatialrelationships . The with eachfeatureis anactionto beusedwhenit is encountered Associated . entersthe infonnationin the mapmanually Actually, this map system designer modelsinceit ties task specificactions is moreof a plan thanan environmental - aninterestingtwist on the useof environmental conditions with recognizable not , wherethe knowledgestoredfor lateruseis dependent only representations on the stateof the world but the robot' s intentionsaswell.

REPRESENTATIONS 5.3 PERCEPTUAL

The representations we have discussed thus far have been concerned with the where issues: where in the world the robot is located and where it is going . As chapter 2 discussed, there is strong neurophysiological evidence for

RESPONSE

DAMN ARBITER

201

Issues BehavioralSystems for Representational dual cortical pathways: one concerned with spatial issues ( where), the other with recognition (what ) (Mishkin , Ungergleider , and Macko 1983) . We briefly discuss in this section some representational issues as they relate to perception for behavior -based robotic systems. This discussion is a preview of a larger discourse in chapter 7 , but it is important at this point to understand certain aspects of representational use for object recognition . A very large body of work exists on model :'based computer vision , which typically is concerned with object identification based on geometric models. Although this strategy is appropriate for certain classesof problems , traditional geometric models have less utility in the context of behavioral systems. Recall from chapter 3 that perception in the context of behavior-based robotic systems is best conducted on a need-to- know basis. Because perception is strongly related to the actions that the robot needs to undertake, we now briefly review some representational strategies that take into account the robot ' s ability to move within the world and that can reflect its intentional position . These representations are consistent with our notion of action -oriented perception and offer in some sense to try to capture the concept of affordance introduced in chapter 7. The main class of representations we consider for sensory recognition are function based, addressing the problem of recognizing environmental objects that fit specific functions of value to the robotic agent at a particular time . These may include affordance-like functions such as sittable , throwable , provide support, provide - storage- space, or what -have-you . A simple example involves assuming that you have a book in your hand and you need to place it down somewhere safely in a room in which you have never been before. Once you walk in , you might see many candidate surfaces ( shelves, tables, beds, etc.) each of which to some degree will perform that function . How is the functional model represented for book stowability , and how can goodness of fit be measured with each candidate' s environmental features? How can a decision be made as to the best place to put down the book? Functional models are concerned with these kinds of issues. Stark and Bowyer ( 1991, 1994) developed one of the first examples of a functional representation. In this first attempt at a function -based model , no explicit geometry or structure is used. Instead a set of primitives (relative orientation , dimensions, stability , proximity , and clearance) defines functional properties . The flow of control for this system (figure 5.18) first takes as input a three-dimensional description of the object in terms of faces and vertices, then identifies potential functional elements ( such as surfaces), attaches functional

No Test Highes Rank Order Functional Ranke 3D Possible Object Cond Object Shap Element Features Remain Categori Identificatio Analy Catego
202
Chapter5

Figure5.18 . flowfor functional Control interpretation

-and a labelsto theindividual featuresandfinally conducts generate -testevaluation , . to determineif it indeedfits the functionalcriteria in question The main used for Stark and Bowyer' s researchhas been identifying a chair example " . as somethingthat both " is sittable and " provides stable support" An armchair " and " " would add the functionsof providesback support providesarm " In this . , geometricmodelscanbe considered strategyobjectswith no express support the chairsif they meetthe constraints functionalmodelsfor that object ' . impose Stark and Bowyer s work hasyet to be integratedinto a robotic system that to , but it appears hold promiseespeciallyfor behaviorbasedsystems -like perceptual . involve affordance strategies More recently Bogoni and Bajcsy ( 1994a havestudiedfunctionality from , ) . an active perceptionperspectiveFunctionality is relatedto observabilityand to provides a basis for robotic sensors investigateactively to determinethe ' s nature Another . actual object ) study (Budenskeand Gini 1994 provides a . basisfor recognizingdoorwaysthrougha seriesof exploratorymotions Sonar data which can be applied for this purpose are notoriously misleadingand , , of error prone because specularreflections a wide samplingcone and other , , ' artifactsof the sensingprocess Everett1995 . In Budenskeand Gini s work, ) ( that initial sonar readingsindicate a functional passageway is potentially a sensormotor activitiesbefore door, but this is confirmedby otherinvestigatory the robot navigatesthrough it . Theseactivities include positioning the robot , adjacentto the door opening having the robot move back and forth in front of the openingto confirm its size and position, and centeringthe robot in the . openingprior to passageThis information is then tied to a behaviorbased /actuators LSAs andthe referredto aslogical sensors robot, using behaviors ), (

Issuesfor BehavioralSystems Representational passagecompleted . The doorway per se is not directly represented; rather, the ' configuration of actions determines the doorway s function . In research at the University of California at Berkeley , Stark developed the theory of scanpaths another active representational strategy potentially , suitable for use in behavior -based robotic systems ( Noton and Stark 1971; Stark and Ellis 1981) . In this theory of object recognition , models are created by the paths taken by the eye as its foveal region moves about the entire object . The connectivity of the focusing points and the order in which they occur constitute the scanpath . This ordering , which can be modeled on a computer by creating a network of visual features, can provide guidance as to where to focus ' perceptual processing to distinguish one object from another. Stark s work has yet to be tied to robotic systems but provides a clear alternative strategy to geometric model- based recognition for use in behavioral control systems. There is much more to be said on the role of perceptual activity in the context of behavior-based robotics . We have touched on only some of the perceptual representation issues here. Chapter 7 investigates more thoroughly perceptual processing and control for this class of robots.

5.4 CHAPTER SUMMARY

. The more predictable the world is , the more useful knowledge representations are. . Two important characteristics of knowledge include its predictive power and the need for the information stored to correlate with the environment in some meaningful way. . Knowledge can be characterized into three primary forms : explicit , implicit , and tacit . . Knowledge can be further characterized according to its temporal durability : . transitory knowledge , which is derived from sensory data and corresponds to cognitive short- term memory . . persistent cognitive maps, which may originate from either a priori knowledge or sensory data and corresponds to long -term memory . . Significant evidence exists from cognitive psychology that mental processing involves various forms of knowledge representation. . Using representational knowledge has several potential drawbacks within behavior -based systems: . The stored information may be inaccurate or untimely . . The robot must localize itself within the representational framework for the knowledge to be of value.

204

Chapter5 . Representational knowledge ' s primary advantage lies in its ability to inject information beyond the robot ' s immediate sensory range into the robotic control system. . Examples of explicit representational knowledge use in behavior- based robots include short- term behavioral memory , sensor derived cognitive maps, - derived representations. and a priori map . Short-term behavioral memory extends behavioral control beyond the robot ' s immediate sensing range and reduces the demand for frequent sensory sampling . This form of representation is tied directly to an individual behavior within the control system. . Grid -basedrepresentations are often used for short-term behavioral memory . The grids are typically either sector-based or regular and have resolutions that ' depend on the robot s environment and source of the grid data. . Grid representations of STM have been used to remember the robot ' s past positions , to buffer sensor readings for wall recognition , and to store observations of obstacles. . LTM maps are either metric or qualitative . Metric maps use numeric values to store the positions of observed events; qualitative maps use relational values. . The notion of distinctive places is central to the use of sensor derived cognitive maps. Locations are remembered that incoming sensor data determine to be unique in some way. . Qualitative maps support general navigational capabilities and can provide behavioral support for moving to a goal , avoiding obstacles, invoking behavioral transitions , localization , and other related activities . . A priori map- derived representations offer the robot information regarding places where it has never been before. The data for the representations may come from preexisting maps, blueprints , ftoorplans , and the like . . Internalized plans inject a priori grid -based map knowledge directly into a behavior-based control system. . The D * method improves on the gradient map strategy used in internalized plans by pennitting efficient sensor updates to the stored world knowledge . . STM and LTM cognitive maps address the " where" aspects of memory ; function -based perceptual representations addressthe " what " aspects . . Function -based perceptual representations are most closely related to the affordance- based approach reactive robotics commonly uses. . Function -based methods are not based on standard geometric methods and require analysis of incoming sensorreadings in terms of what the robotic agent needs to accomplish .

Chapter Hybrid

6 DeUberative / Reactive

A rehiteetures

In preparing for battle I have always found dlat plans are useless but planning is , . indispensable - Dwight Eisenhower . It is a bad plan that admitsof no modification
- Publilius Syrus

' Everybodys got plans . . . until they get hit. - Mike Tyson
Few people think more than two or three times a year ; I have made an international reputation for myself by thinking once or twice a week. - George Bernard Shaw

Objectives Chapter met delibe limitations of betwe ITomodels .To in.reactive understand theinterfac purely purely inmodels istoreactive considered isolation when/and eachdeliber . of these 2 biological .To express syste hybrid studyfor.,Atla cont react thearchitec issues and 3 planners establish recognize interf several and ,espec deliberative 4 several ., -representa study AuR .hybrid RPRS and Planner , eactor

206

Chapter

6.1 WHYHYBRIDIZE ?
We haveseenthat reactivebehaviorbasedrobotic control can effectivelyproduce robust performancein complex and dynamic domains In someways . , however the strongaSsumptions purely reactivesystems that makecan serve , asa disadvantage times. Theseassumptions at include 1. 2. 3. 4. The environment lackstemporalconsistency stability. and The robot' s immediatesensing adequate the task at hand is for . It is difficult to localizea robot relativeto a world model. world knowledgeis of little or no value . Symbolicrepresentational

In some environments however theseassumptions , , may not be completely valid. Purely reactiverobotic systemsare not appropriatefor all robotic applications . In situationswhere the world can be accuratelymodeled uncertainty , is restricted and someguarantee exists of virtually no changein the , world during execution(such as an engineered assemblywork cell), deliberative methodsare often preferred since a completeplan can, most likely, , be effectively carriedout to completion In the real world in which biological . function, however the conditionsfavoring purely deliberativeplanners , agents generallydo not exist. If roboticistshope to havetheir machinesperforming in the sameenvironments we, ashumansdo, methods behaviorbased that like , reactivecontrol arenecessary . Many researchers , however that hybrid systems feel , of incorporatingboth deliberativereasoning behaviorbased and capable executionare neededto deliver the full potential of behaviorbasedrobotic . systems We saw in chapter5 that introducing various forms of knowledgeinto a robotic architecturecan often make behaviorbasednavigationmore flexible and general Deliberative systemspermit representational . knowledgeto be usedfor planning purposesin advanceof execution This potentially useful . knowledgemay take severalforms: . Behavioral and perceptualstrategiescan be represented modules and as . , configuredto matchvariousmissionsand environmentsaddingversatility . A priori world knowledge whenavailableand stable canbe usedto configure , , or reconfigure thesebehaviors efficiently. . Dynamically acquiredworld modelscan be usedto preventcertainpitfalls to which non-representational methodsare subject . / Hybrid deliberativereactive robotic architectureshave recently emerged of traditional AI symbolic methodsand their use of abcombining aspects

207

Hybrid Deliberative / ReactiveArchitectures

stract representational , knowledge but maintainingthe goal of providing the . , robustness and flexibility of purely reactive systems Hybrid , responsiveness architectures of reactivecontrol systems basedon permit reconfiguration availableworld knowledgethroughtheir ability to reasonover the underlying behavioralcomponents . Dynamic control systemreconfiguration basedon deliberation overworld models is an importantadditionto the overall (reasoning ) of . competence generalpurposerobots such a hybrid system however requirescompromisefrom both , , Building endsof the robotic systems , spectrum(section1.3). Furthermorethe natureof the boundarybetweendeliberationand reactiveexecutionis not well understood at this time, leadingto somewhat decisions . arbitrary architectural 6.2 BIOLOGICAL EVIDENCE IN SUPPORT OF HYBRID SYSTEMS Psychologicaland neuroscientificmodels of behavior provide an existence of proof for the success an integrativestrategyinvolving elementsof deliberative and behaviorbasedcontrol. Flexibility in our use of the reasoning modelsscientistsin other fields havedeveloped important however since is , , we, as roboticists are concernedprimarily with creatingfunctioning autonomous , agentsthat may have somebehavioraloverlap with their biological with reproducingtheir control andexecution , counterpartsbut not necessarily verbatim . strategies Just as many psychologists movedfrom behaviorism( Watson 1925 Skin; ner 1974 to cognitivepsychology( Neisser 1976 asan acceptable ) ) description of humaninformation processing researchin the use of hybrid systemshas , to forwardedby this schoolof thought The . expanded include many concepts . ) experimentalevidenceis compelling Shiffrin and Schneider( 1977 haveindicated the existence two distinct modesof behavior willed andautomatic of : . NormanandShallice( 1986 havemodeledthe coexistence two distinct systems of ) concerned with controlling humanbehavior One systemmodels" automatic . " behaviorandis . closelyalignedwith reactivesystemsThis systemhandles automaticaction executionwithout awarenessstartswithout attention , , . and consistsof multiple independent threads(schemas The ) parallel activity secondsystemcontrols " willed" behaviorand providesan interfacebetween . deliberateconsciouscontrol and the automaticsystem Figure 6.1 illustrates this model. the This researchcharacterized tasks requiring willed control in humans deliberate attentionalresources : involving

208

Chapter6

] (vertical control threads)

Figure 6.1 Model for integrated automaticandwilled behavior(after Noonanand Shallice 1986 . )

. planning decision or I making . troubleshootin2

. novel or poorly learned actions . dangerous or difficult actions . overcoming habit or temptation ' Other motor tasks are typically automatic and occur without the use of attention . Their modeling incorporates a contention scheduling mechanism for . coordinating the multiple active motor schemas Higher -level deliberative processes involving attention alter the threshold values for schemas ( behaviors), dynamically changing the interplay between them. Psychological support for schema use in this strictly horizontal manner is well established ( Schmidt 1975) . The Nonnan - Shallice model incorporates aspects of both vertical and horizontal control threads. The horizontal control threads are used in a similar manner in the subsumption architecture ( Brooks 1986) . Deliberative influence is introduced when multiple horizontal behaviors are mediated by vertical threads that interconnect the various behaviors and allow for their dynamic modulation as a result of attentional resources, such as planning ,

209

/ Reactive Architectures HybridDeliberative eventstrigger the schemas themselves attentional but , troubleshootingetc. Perceptual es process modulatethem. This provides a coherentpsychological modelfor integratingmultiple concurrent behaviors controlled by higher level . processing The Norman Shallicemodelpoints out severalconnections betweendeliberate andautomaticcontrol: . Automatic schemas modulated attentionarisingfrom deliberatecontrol are by . . Schemas tasks competewith eachother . (behavioral ) . Schema selectionis deliberatecontrol' s principal function. Vertical threads . providethe selectionmechanism . Neuropsychological are with this model. experiments consistent The evidence of attentionalsystem supportingthe existence a distinct supervisory is considerableThe model lacks a mechanism which the deliberative . by is . ; process conducted this is left for othersto elucidate Evenif this theoryultimately fails to explainthe basisfor humanpsychological motor behaviorand useful in its own right as a basis , planning the model may nonetheless prove for integratingdeliberativeandbehavioralcontrol systems robots in .

' 6.3 TRADITIONAL LIB ERAnVEPLANNERS DE

Deliberativeplannersare often alignedwith the hierarchicalcontrol community within robotics (Hierarchicalcontrol is alsoreferredto asintelligent control . ; seesection 1.3.1.) Hierarchicalplanningsystems typically sharea structured and clearly identifiablesubdivisionof functionality relegated distinct to modulesthat communicate with eachotherin a predictable predetermined and program manner Numerousexamplesillustrate this deliberative . /hierarchical e.g., Albus, McCain, and Lumia 1987 Saridisand Valva ; planning strategy( nis 1987 Meystel1986; Keirseyet ale1984 Ookaet ale1985. A generalized ; ; ) modelappears figure 6.2 in . A typical subdivisionof functionality depends both the spatialplanning on ' and temporalconstraints At a hierarchicalplanners highestlevel, the . scope most global and least specificplan is formulated The time requirements . for . down the planning producingthis plan are the least stringent As one proceeds narrower focusing on smaller regionsof , , hierarchy the scopebecomes the world but requiring more rapid solutions At the lowestlevels rapid real. , time response required but the planneris concerned is , only with its immediate

U~cw ~~. ~

210

Chapter6 HIERARCHICAL PLANNER Strategic . Global Planning Tactical Intermediate Planning Short-Term Local Planning

WORLD MODEL Global Knowledge

TIME HORIZON
Long-Term

Local World Model

Actuator Control

Immediate Sensor Interpretations Real-Time

ACTIONS Figure 6.2 Dellberative /hierarchcal planning i .

SENSING

and has lost sight of the " big picture." Meystel ( 1986 has developed ) surroundings a theory for hierarchicalplanningthat emphasizes significanceof the . scopeandinvokesthe conceptof nestedcontrollers Hierarchicalplannersrely heavily on world models can readily integrate , and . world knowledge and havea broaderperspective scope Behaviorbased , control systems on the other hand afford modular development real-time , , , robustperformance within a changingworld, and incremental growth, and are with arriving sensorydata Hybrid robotic architectsbelieve . tightly coupled es that a union of the deliberativeandbehaviorbasedapproach canpotentially , yield the best of both worlds. H donepoorly, however it can yield the worst of both worlds. The central issue then becomeshow to developa unifying architecturalmethodologythat will ensurea systemcapableof robustrobotic of plan executionyet take into accounta high-level understanding the nature of the world anda modelof userintent.

211

I Architectures Hybrid Deliberative Reactive

6.4 DE ERA Lffi

T I PLANORNOTTOPLAN TO ON: ?

The integrationof knowledgebaseddeliberationandreactivecontrol requires the confrontationof manydifficult problems Eachof thesemethods . address es differentsubsets thecomplexitiesinherentin intelligentrobotics The hybrid of . ' systems architect contendsthat neither approachis entirely satisfactoryin isolation but that both must be taken into accountto producean intelligent , . robust and flexible system , The hierarchicalapproachis best suited for integrating world knowledge and user intent to arrive at a plan prior to its execution Replanningwith this . method however at levels where sensorydata is mergedinto world models , , is cumbersome best Deliberativeplanning without considerationfor the at . difficult issues planexecution leadto restricted of can within very narrow usage ' domains(i .e., the ecologicalniche is extremelysmall and focused problem ) andextremelybrittle robotic systemsA robot musthavethe ability to respond . that occur within rapidly and effectively to dynamic and unmodeledchanges its world. If a purely deliberativesystemattemptsto model and preplanfor all eventualitiesit risks becomingso boggeddown that the planningprocess , neverterminates thequalificationproblem (seebox 6.1). It is alsounsafefor a ( ) robot to makegrossassumptions aboutthe world that do not reflectits dynamic nature .

Boxeifan 1proble 6alliitThu ..themha "ab Thefs.to ispos relate tomo aand neve -"sin w ? qualific nd qu 'smay stream to"fail areal mak mor res Qualifi plan utility "( to less.wh .amany There are)be toowe w ap genera justrec -porl enume in,dom advan forha any 'sit ca adequa qualify plan applic

The reactiveapproach on the other hand is well situatedto deal with the , , of sensory databut is lesseffectivein integratingworld knowledge . immediacy ' A clear cut distinctioncanbe seenin the hierarchicalplanners heavyreliance on world models(either a priori or dynamicallyacquired as compared the to ) avoidancein most reactivebehaviorbasedsystemsof world representations entirely. When reactivebehaviorbasedsystemsare consideredin isolation, robustness gained at the expenseof some very important characteristics is : . , flexibility and adaptability The issuesof action andperceptionare addressed

212

Chapter6

but cognition is ignored often limiting theserobots to mimicking low-level , life forms. Hybrid systemresearchassumes representational that knowledge is necessary enhance extendthe behaviorsof thesemachines to and into more domains This includesthe incorporationof memoryand . meaningfulproblem of . dynamic representations the environment Dynamic replanning must be affectednot only in a reactivemannerbut alsoin the contextof a moreabstract ' the plan, one representing robot s goals and intents at a variety of planning levels The research . issues thesedesigners not centeron reactiveversus for do deliberativecontrol but rather on how to synthesize preplanned effectively a control regimethat incorporates both methodologies . The terms signifying each of the two major components thesehybrid of architecturesvaries widely. Lyons ( 1992 uses planner and reactor Malcolm . ) and Smithers ( 1990 prefer cognitive and subcognitivesystems with ) , the cognitive componentperforming high-level functions such as planning ' and the subcognitive and . portion controlling the robot s sensors actuators In this book we generally use deliberativeand reactiveto distinguish the two . systems The central issue in differentiating the many approach to hybrid architectures es discussedin this chapter focuseson interface design What is the : for the subdivisionof functionality? How is coordination appropriateboundary effectively carried out? This is one of the most interestingand pressing research areas intelligent roboticstoday in . threedifferent waysin which planningandreaction ) Lyons ( 1992 describes canbe tied: . Hierarchicalintegrationof planningandreaction Deliberativeplanningand : reactiveexecutionareinvolvedwith differentactivities time scalesandspatial , , . Hencea multilevel hierarchicalsystemcanbe structured that integrates scope both activities (panel(A ) in figure 6.3). Planningor reactingdepends the on situationat hand In manyways this is closely alignedwith the traditionaldeliberative . , with one fundamental distinction: the higher deliberative , approach level(s) are epistemologically distinct from the lower, reactiveone s), that is, ( the natureandtype of knowledgeandreasoning distinct. is . Planningto guide reaction Another alternativemodel involves permitting : for . planningto configureand set parameters the reactivecontrol system Execution ' occurssolely underthe reactivesystems auspiceswith planningoc, , curring both prior to and concurrentwith execution in somecases projecting the outcomeof continuouslyformulatedplans and reconfiguringthe reactive systemasneeded (panel(B) in figure 6.3). . Coupledplanning reacting Planningand reactingare concurrentactivities : , eachguiding the other (panel(C) in figure 6.3).

213

/ Hybrid Deliberative Reactie Architectures ~ More Deliberative

~ I

I I Planner Deliberation Projection ~ Behavioral Advice Configurations Parameters [ ~~ ] : ~ =~ J

Level 2 Level1 Level 0 MoreReactive (A)

(8)

(C)

Figure6.3 deliberative /hierarchical . Typical planning strategies

6.5 LAYERING
Oneoutcome a 1995workshopon robot architecturesHexmooret aI. 1995 of ( ) wasthe observation a multilayered hybrid architecture that comprisinga toplayer planning systemand a lower-level reactivesystemis emergingas the architectural ) designof choice(HexmoorandKortenkarnp1995 . It wasfurther observedthat the interfaceor middle layer betweenthe two components of suchan architecture the key function, linking rapid reactionand long-range is . planning look toward a synthetic integrativeapproach that , Hybrid systemdesigners both of theseparadigms(reactionand planning to the issuesof robot ) applies control, using eachwheremost appropriateAfter the decisionhasbeenmade . that both deliberativeand reactivefunctionality are importantfor a particular , application the questionarisesas to how to effectively partition thesefunctions . In general two layersareneeded a minimum: oneto represent at deliberation , andthe otherreactivity Anothercommonapproach . involvesintroducing an explicit third layer concerned with coordinatingthe two componentsSection . , 6.6 looks at specificinstances thesehybridizedarchitectures gives of and of , , , examples both. Indeed in somecasesfurtherresolutionis added producing evenmorelayers The bottom line, however is that deliberationandreactivity . , needto be coordinatedandthe architectdecides whereandhow to implement , this function.

214

Chapter6

HYBRID 6.6 REPRESENTATIVE ARCIDTECTURES

Four principal interfacestrategies in evidence the varioushybrid architectural are for : designs . Selection Planning is viewedas configuration The planning component : . determines behavioralcompositionandparameters the usedduring execution . The planner may reconfigurethem as necessary because failures in the of . system . Advising: Planningis viewedasadvicegiving. Theplannersuggests changes that the reactivecontrol systemmayor may not use This is consistent . with the " " asadvice view (Agre andChapman1990 in which plansoffer courses ) plans of actionsbut the reactiveagentdetermines whethereachis advisable . . Adaptation Planning is viewedas adaptation The planner continuously : . altersthe ongoingreactivecomponentin light of changingconditionswithin the world andtaskrequirements . . PostponingPlanning is viewedas a leastcommitment : . process The planner defersmaking decisionson actionsuntil as l ~te as possible This enablesrecent . sensordata by postponingreactiveactionsuntil absolutelynecessaryto , , a moreeffectivecourseof actionthan would be developed an initial if provide at . . plan were generated the beginning Plansareelaborated only asnecessary This sectionpresents four major hybrid architectureseachof which typifies , one of thesestrategies AuRA for selection Atlantis for advising Planner : ; ; Reactoras adaptation and PRS as postponementA surveyof severalother . ; is . hybrid architectures then presentedJudgingfrom the high level of activity, the hybrid approach currently a particularly importantresearch is topic. Note the distinctionsin how deliberationandreactivity areinterfaced as , especially . thesearehallmark characteristics eachapproach for
6.6.1 AuRA

Arkin ( 1986 1987b was amongthe first to advocatethe use of hybrid deliberative , ) -based control systemswithin ) ) (hierarchical and reactive(schema the AutonomousRobot Architecture (AuRA). Incorporatinga conventional plannerthat could reasonover a flexible and modularbehaviorbasedcontrol could be constructed , system Arkin found that specificrobotic configurations that integratedbehavioral perceptual and a priori environmental , , knowledge : ( 1990b. Hybridization in this systemarisesfrom two distinct components ) a deliberativehierarchicalplanner basedon traditional AI techniques and , ,

215

Hybrid Deliberative / ReactiveArchitectures

a reactivecontroller basedon schematheory (Arbib 1992 . Arkin ' s was the , ) first robot navigational system to be presentedin this integrative manner ) ( 1989d. of . Figure 6.4 depictsthe components AuRA schematicallyAuRA hastwo : componentsa hierarchicalsystemconsisting major planning and exeCution of a missionplanner spatialreasonerand plan sequencer , , coupledwith a reactive . controller In the style of a traditional hierarchical , the schema system ; ; planningsystem(Albus, McCain, andLumia 1987 Meyste11986Saridisand with Valvanis1987 the highestlevel of AuRA is a missionplannerconcerned ), within which it establishing high-level goalsfor the robot and the constraints constructed date the missionplanto must operate In AuRA-basedsystems . , . ner has actedprimarily as an interfaceto a humancommanderThe spatial referred to as the navigator (Arkin 1987b usescartographic reasoner originally , ), of knowledgestored in long-term memory to constructa sequence . path legs that the robot must executeto completeits mission In navigational the first implementationof AuRA, this was an A * planneroperatingover a meadowmap (hybrid free space /vertex graph representationArkin 1989c. ( ) ) eachpath The plan sequencer , referredto asthe pilot in earlierwork, translates . into generates a set of motor behaviorsfor execution leg the spatialreasoner was a rudimentaryruleIn the original implementation the plan sequencer , as based . .en system More recentlyit hasbe implemented a finite statesequencer , , ) (Mackenzie Cameron and Arkin 1995 . Finally, the collection of behaviors , is (schemas specifiedandinstantiated the plan sequencer then sentto the ), by . robot for execution At this point, deliberationceasesand reactiveexecution , . begins for The schema manageris responsible controlling and monitoring the behavioral es ) process at run time. Eachmotor behavior(or schema is associated of providing the stimulusrequiredfor that with a perceptualschema capable . particularbehavior This action orientedperceptionis the basisfor this form -basednavigation(Arkin 1990a. As described section4.4, each in of behavior ) to a vector in a manneranalogous the potential behaviorgenerates response fields method The schemas . , transmittingtheir results operateasynchronously theseinputsandtransmits to a process move robot) that sumsandnormalizes ( . to the low-level control systemfor execution them control system(testedonly in simulation to Within AuRA, a homeostatic date is interwovenwith the motor and perceptualschemas Arkin 1992c. ) ( ) transducersprovide information Internal sensors such as fuel level and temperature , , network monitoredby behaviorscontainingsuitable over a broadcast 's . changethe overall motor response receptors Theseinternal messages

216

6 Chapter

,-- - - - - - - - - - , I - - - - - - - - - - ,I I I : Plan RecognitionUser : : Intentions : Profile : I : User I I I I I I I I I I I I I I . IL . I : Spatial GoalsI IS I patiaearning I I I I I I I I : I I I I I I I A .SmI : I Mission I I pportunl I I I I Alterations : I I I I I I I I I I I -line I : Teleautonomy : On : I : I Adaptation I I I.L - - - - - - - - - -I I ______- - - - -I L

Learning

User Input

Hierarchical Component

Reactive Component

Figure 6.4 . High-level AuRA schematic ' ' performance by altering the behaviors and internal parameters relative strengths in an effort to maintain balance and system equilibrium ( homeosta sis) . Chapter 10 discusses homeostatic control further . Once reactive execution begins, the deliberative component is not reactivated unless a failure is detected in the reactive execution of the mission . A typical failure is denoted by lack of progress, evidenced either by no motion or a time -out. At this point the hierarchical planner is reinvoked one stage at a time , from the bottom up , until the problem is resolved. First , the plan sequencer attempts to reroute the robot based on information obtained during navigation and stored in STM . Original implementations used sonar maps produced by the Elfes -Moravec algorithm for spatial world modeling (Elfes 1986) . If for some reason this proves unsatisfactory (e.g ., the route is completely blocked within this local context ), the spatial reasoner is reinvoked and attempts to generate a new global route that bypasses the affected region entirely . H this still fails to be satisfactory, the mission planner is reinvoked , informing the operator of the difficulty and asking for reformulation or abandonment of the entire mission. Modularity , flexibility , and generalizability as a result of hybridization constitute Au RA' s principal strengths. The value of each aspect has been demonstrated both in simulation and on real robotic systems.

217

/ Architectures Hybrid Deliberative Reactive

AuRA is highly modularby design Components the architecture be . of can with othersin a straightforward manner This is particularlyuseful in . replaced researchSomeexamples . include . A specialized for task whereboxes missionplannerdeveloped an assembly . are pushedtogetherinto a specifiedarrangementThis planner was ported . to a Denning mobile robot that competedin the 1993American Association for Artificial Intelligence (AAAI ) mobile robot competition Stroulia ( 1994 . ) further extendedthe plannerto learn and reasonover more generalplanning tasks . . The original A * spatialreasoner beenreplacedwith Router(Goel et at. has 1994 a multistrategyplanner Routermodelsnavigableroutesas topological . ), links betweennodesinsteadof the metric meadowmap representation used . The systemwastestedon a Denningmobile robot that success previously fully navigatedfrom room to room and down corridorsin an office and laboratory building. . Perceptual schemas havebeenexpanded incorporatespecializedaction to orientedsensor fusion methods(Murphy and Arkin 1992 (section7.5.7). Because ) of the recognitionthat in manycases are multiple sensorsources better than individual ones specialized weredeveloped fusedatawithin to , strategies the contextof action orientedperception DempsterShaferstatisticalmethods . the basisfor evidentialreasoning Murphy 1991 . ( ) provided . Theoriginal rule-based has with a temporalsequencer plan sequencer beenreplaced , , (Arkin and MacKenzie1994 basedon FSAs (MacKenzie Cameron ) and Arkin 1995 . The FSA is an expression a plan, in which eachstaterepresents of ) a specific combinationof behaviorsthat accomplishone step of the task. Transitionsare madefrom one stateto anotherwhen significantperceptual eventstrigger them. Anotherstrengthof AuRA is the flexibility it providesfor introducingadaptation and learning methods Chapter8 will discusstheseand other methods . further. In early implementations AuRA, learningaroseonly from STM of of . spatialinformationusedfor dynamicreplanning Sincethen, a varietyof learning havebeenintroduced including , techniques . on-line adaptationof motor behaviors using a rule-based methodology (Clark, Arkin , andRam 1992 ) . casebased methods providediscontinuous to reasoning switchingof behaviors basedon the recognitionof new situations(Ramet at. 1997 ) . geneticalgorithmsthat configurethe initial control systemparameters efficiently (Ram et at. 1994 and allow a robot to evolve toward its ecological ) nichein a given taskenvironment

218

Chapter6

Au RA' s generalizabilityto a wide range of problemsis anotherstrength . Variousarchitecturalcomponents havebeenappliedin a variety of domains , including . manufacturing en onments(Arkin et al. 1989 Arkin andMurphy 1990 . ; ) ~ . three dimensional asfound in aerialor undersea domains(Arkin navigation 1992a. ) . indoor and outdoornavigation(Arkin 1987b. ) . robot competitions(Arkin et al. 1993 Balch et al. 1995 . ; ) . vacuuming( MacKenzie Balch 1993 . and ) . military scenariosMacKenzie Cameron andArkin 1995 Balch andArkin , ( , ; 1995 . ) . mobile manipulation(Cameron al. 1993 . et ) . multirobot teams(Arkin 1992b Balch andArkin 1994 . ; ) AuR s major strengthresultsfrom the power of weddingtwo distinct AI A' : . paradigms deliberationand reactivity AuRA provides a framework for the conductof a wide rangeof robotic researchincluding deliberativeplanning , reactivecontrol, homeostasisaction orientedperception and machinelearning , , . It hasbeenmotivatedbut not constrained biological studies drawing , by . insight whereveravailableasa guidelinefor systemdesign Au RA' s strengthslie in its modularity which pennits ready integrationof , new approach to variousarchitecturalcomponentsflexibility , as evidenced es ; and by the easeof introduction of variouslearning methodologies novel behaviors ; generalizability demonstrated its applicability to a wide rangeof , by domains including robot competitions amongothers and most importantly , , ; , use of hybridization to exploit the strengthsof both symbolic reasoningand reactivecontrol.
6.6. 2 Atlantis

At the Jet PropulsionLaboratory(JPL) Gat ( 1991a developed three level a ) a , hybrid system Atlantis, that incorporates deliberatorthat handlesplanning andworld mbdeling a sequencer handles that initiation andterminationof low, level activitiesand address reativesystemfailures to completethe task, and es a reactivecontroller chargedwith managingcollectionsof primitive activities is and . None (figure 6.5). The architecture both asynchronous heterogeneous of the layers is in chargeof the others and activity is spreadthroughoutthe , architecture .

219

/ Architectures Hybrid Deliberative Reactive

SENSORS
Status
A

ACTUATORS

Results

Invocation

Figure6.5 TheAtlantis architecture . Atlantis' s control layer is implemented ALFA (Gat 1991b a LISP-based in ), usedto programreactivemodulesconfiguredin networks programlanguage connected communication via channelsALFA is mostcloselyrelatedto Kael. ' s Rex 1987 a circuit-based ( ), bling ( language section3.2.4.2). This systemwas initially testedon Tooth (figure 6.6, panel(A , a small precursorto the Mars microrovers usedfor NA SA s Pathfinder ' program(ShirleyandMatijevic 1995 ) (figure 6.6, panels(B) and(C , anda RealWorld Interface( RWI basefor indoor ) . navigationalexperiments The sequencinglayer of Atlantis is modeled after Firby' s RAPs (section 3.1.3). Conditional sequencing occurs upon the completion of various subtasks thedetectionof failure. In particular the notionof cognizant or , failure is introduced(Gat andDorais 1994 referingto the robot' s ability to recognize ), on its own when it has not or cannotcompleteits task. Monitor routinesare addedto the architectureto determineif things are not going as they should and then interrupt the systemif cognizantfailure occurs Often thesemonitor . routinesare very task specific such as checkingalignmentconditionswhen , conductingwall following, but they can be more general suchas a time-out , for the overall completionof a task. ' Deliberationoccursat the sequencing ( layer s request Gat 1992 . The deliberator ) consists traditionalLISP-based planningalgorithmsspecificto the of AI ' task at hand The planners output is viewed only as adviceto the sequencer . followed or implemented verbatim . layer: it is not necessarily

TIVE

ERA

DELIS

220

Chapter6

(A)

) (B 6.6 Figure

221

/ Architectures Hybrid Deliberative Reactive

(C)
Figure 6.6 (continued )

4 . ,( (A) ToothB) Rocky, andC) Sojourner (

Atlantis' s importantfeatures : Summarizing . Atlantis is a three layeredarchitectureconsistingof controller sequencer , , anddeliberator . . Asynchronousheterogeneous . , reactivity anddeliberationareused

Design in Atlantis proceeds from the bottom up : low - level activities capable of being executed within the reactive-controller level are first constructed. Suitable sequencesof these primitive behaviors are then constructed for use within the sequencing level , followed by deliberative methods that assist in the decisions the sequencer makes. Experiments have been performed on a large outdoor JPL Mars rover testbed called Robby (figure 6.7 ) (Gat 1992), which success fully undertook various tasks in rough outdoor terrain . The primitive activities complex navigational used in the reactive controller were based on Slack ' s NATs ( section 3.3.2) and were guided by a strategic plan constructed by the deliberator. The sequencer was then able to abandon intermediate - level navigational goals if they became untenable as noted by advice from the deliberator.

222

Chapter6

Figure6.7 . , Robbya JPLMarsroverprototype . The resultsof deliberationareviewedasadvice not decree . , . Classical AI is merged effectively with behaviorbased reactive control methods . . Cognizantfailuresprovide an opportunityfor plan restructuring . . The systemhas been exercisedsuccess on both indoor and outdoor fully robotic systems .
6.6. 3 Planner -Reactor Architecture

, ) Lyons and Hendriks ( 1992 1995 forward the PlannerReactorarchitecture as anothermeansfor integrating planning and reactivity Their philosophy . advocatesthe use of a planner as a mechanismto continuously modify an executingreactivecontrol system Figure 6.8 depicts this approach The . . is in essence executionmonitor that adaptsthe underlying behavioral an planner ' control systemin light of the changingenvironmentand the agents . underlyinggoals

223

I Architectures Hybrid Deliberative Reactive

ACTION

1 I I I -- .I

- - - - - - - - - - - '

I I I I : I I I I I 1

WORLD : I . I SENSINGI I- - - - - - _ _ _ _ _ _

Figure 6.8 PlannerReactorarchitecture .

The RS model discussed section3.2 4.1, is usedboth to model and to in . , . implement the reactor component It is assumedthat a suboptimalreactor ' may be presentat any time and that the planners goal is to improve the of , performance the reactor at all times. Loosely speaking this is a form of anytimeplanning, wherea significantly suboptimalsolution may be initially chosenthenimprovedon during execution .

Situationsprovidethe frameworkfor structuringsetsof reactions They can . be hierarchicallydefinedand often denotethe statethe robotic agentis currently ' in regardinga task. For the primary task studiedin LyonsandHendriks , , ( 1993 work parts assemblya situationalhierarchycan be structuredas depicted ) in figure 6.9. Here the situation where the robot needsto build kits , consistsof variousconstituentsituations eachof which may in turn consistof , further situationalspecificationsThis hierarchyis not unlike the tasklsubtask . hierarchies suchas Noah (Sac developed traditional AI planningsystems by erdoti 1975 but differs in that the situationsspecifybehavioralstructures for ), usein the reactorandnot specificrobotic commands .

such Wellman

manner and Dean

critical . cf (

time

a time

in and , over

answers execution increases for plan approximate available is available provide the plan a of point planners quality . any ) : At The 1991 Anytime that . .

224

Chapter6

/ Y FINDTRA

BUILDKITS """ :eU FFERPARTS : IHANDLETRA Y : ------ -------: ", ~ r PLACEPARTS Y MOVETRA GEl TRAY

------------__ - , ~ J-PART A :HANDLE: - : :

B PART HANDLE 1 1- - - - - - - --- - - - - ----, PART '' C HANDLE 1 -

The situationsthemselves also encoded are . using RS formalisms Planning is viewed as a form of adaptation( Lyonsand Hendriks 1994 . A reactorexecutes ) under a set of operatingassumptionsIf any assumptions violated . are , regardingthe utility of a particularreactorconfiguration the plannermodifies ' the reactors control systemto removethe assumption violation. If the violation occursas a result of environmental , changesthe strategyis referredto as relaxation Plannerdirectedrelaxationof assumptions . can forced assumption -level goals(e.g., from userinput). The alsooccurbecause a changein high of usedwithin the PlannerReactorarchitecture generallyhighly are assumptions domainspecific(i.e., strongknowledge. )

.Strongcan domai to involves Knowledgeand peculiar andgeneralproble hasinformation littlethata dom or utility has no .beparticul .Weak used information Knowledge many . is broad across utility
work cell, someof these For partsassembly a robot in a manufacturing by -Reactorarchitecture include within the Planner assumptions . Partquality: Eachpart meetsthe necessary for criteria or specifications use . in the assembly . Non-substitutabilityof parts Eachpart hasonly onetype. : . No partsmotion: Partsdo not moveoncedeliveredinto the work space .

225

/ Architectures Hybrid Deliberative Reactive . No downstream disturbance: Subsequent manufacturing processes are always to receive the assembled parts. ready . Filled tray : All the parts are delivered to the work cell . . Tray disturbance: The tray is not moved after arrival . . Parts homogeneity : Parts arrival is evenly distributed . Clearly , in the real world , violations of these assumptions are not only possible but likely . Each assumption has a monitor associated with it during run time to ensure its validity . If , for whatever reason, an assumption violation is detected, the planner relaxes the assumption and adapts the control system to deal with the new situation . These violations often occur because of environmental factors ' beyond the robot s control . The planner can reinstate assumptions later, once the original situation has been restored, along with a reactor reconfiguration and reinstantiation of a suitable assumption monitor . Figure 6.10 depicts the flow of control in this architecture. This process is recursive, as an adapted reactor can be further adapted by the planner. A variation of the Planner- Reactor architecture has been developed for planning and controlling a multifingered robotic hand (Murphy , Lyons , and Hendriks 1993) . The deliberative planner is referred to as the grasp advisor and has an associated grasp reactor. Grasp selection is based (ideally ) on the task requirements, the feasibility of acquiring the part using the proposed grasp, and the stability afforded the part once grasped in that manner. The initial implementation , however, is concerned only with the stability criterion . Typical behavioral components for a reactive grasping system include find -objects, grasp-objects, and avoid - obstacles, which are all self- explanatory in function . The deliberative grasp advisor, using information obtained from environmental knowledge such as part information obtained through vision , communicates global constraints to the reactor, which then blases the actual grasp strategy used for initial contact with the part . This example task does not currently use assumptions within the grasp advisor in the same way that the kitting assembly system does but nonethelessexemplifies how deliberation and reactivity can be effectively integrated.

, Summarizing the key featuresof the PlannerReactormethodologyare as follows: . Deliberationand reactivity are integratedthroughasynchronous interaction of a planneranda concurrent reactivecontrol system . . Planningis viewedasa form of reactoradaptation . . Adaptationis an on-line process ratherthan an off -line deliberation . . Planningis usedto removeerrorsin performance whenthey occur .

226

Chapter6

REACTOR ADAPTED BY PLANNER AND ASSUMPTIONS RELAXED REACTOR PERFORMANCE WITH MONITORING RESTOREHALT INITIAL REACTOR
Figure6.10 -based . flow FSArepresenting of control for assumption planning
. The reactor undergoes situationally dependent on-line perfonnance improvement . . The basic techniques, tested in both assembly work cell tasks and grasp planning for a robotic hand, are believed applicable to a broad range of applications , including mobile robot navigation and emergency response planning . 6.6.4 The Procedural Reasoning System

The ProceduralReasoning ), , ) System (PRS (Georgeffand Lansky 1987 provides an alternativestrategyfor looking at the integrationof reactivity and of . deliberation Reactivityin this systemrefersto the postponement the elaboration -commitmentstrategy . of plansuntil it is necessary type of least ,a

227

/ Architectures Hybrid Deliberative Reactive

Figure 6.11 The Procedural ) reasoning system( FRS.

-available Asomakin leastcorre cinforma ommitm defers adie decis itis ne abs toindlus do ,tonee .strategy Theuntil make ais dec ass neces tolate forb become proces reduc . die

In fRS , plansarethe primary modeof expressing action but theseplansare , determinedin reactionto the current situation Previouslyformulated . continuously executioncan be interruptedand abandoned any at plansundergoing ' s beliefs desires andintentionsareall used time. Representations the robot of , , to formulatea plan. The plan, however represents robot' s desiredbehaviors the , ' . insteadof the traditionalAI planners outputof goal states be achieved to 6.11 depictsthe overall PRS architecture The interpreterdrives system . Figure execution carrying out whateverplan is currently deemedsuitable As . , new beliefs desires or intentionsarise the plan may change with the interpreter , , , , . , handlingthe plan switching A symbolicplan alwaysdrivesthe system however so it is not reactivein the nonnal sense tight sensorimotor execution of , pair , but it is reactivein the sensethat perceivedchangingenvironmental conditionspermit the robotic agentto alter its planson the fly. The systemwastestedanddeveloped SRI' s robot Flakeyfor usein office on tasks(figure 6.12). UM -PRS ( Lee et al. 1994 is a later variation ) navigation

228

Chapter6

Figure6.12 and .) . of courtesy KurtKonolige SRIInternational Flakey(Photograph

229

/ Hybrid Deliberative Reative Architecture ~ ~

of this FRS systemthat has beenappliedto the DefenseAdvancedResearch Project Agency UnmannedGround Vehicle ( DARPAUGV) Demo n project for outdooroff -road military scoutingmissionsusing HMMWVs. We revisit this systemas a pieceof the DARPA UGV Demo n programin chapter9.

6.6.5 Other Hybrid Architectures

We now survey other efforts in the development of hybrid deliberative/reactive architectures. The solutions being explored are diverse, especially in regard to where deliberation should end and reactivity begin and whether planning should be viewed as selection, advising , adaptation, postponement, or something else. . SSS (Connell 1992) : SSS, developed at the IBM T.J. Watson Research Center , is a hybrid architecture that descended directly from the subsumption architecture (section 4.3) . The letters in SSS stand for each of its three layers : servo, subsumption , and symbolic . The interface between the servo layer and the symbolic is not particularly new: together they provide behavioral modularity and flexibility to the underlying servomotor controllers by providing parameters and set points for the servo loops in the same manner as subsumption . SSS' s novelty lies in its use of world model representations, which are viewed as a convenience, but not a necessity, for certain tasks. The symbolic (deliberative ) layer provides the ability to selectively turn behaviors on or off as well as provide parameters for those that require them. Once the behaviors are configured , they continue to execute without any intervention of the symbolic level. Restating, the symbolic level predetermines the behavioral configuration used during execution. The system was tested on a small mobile robot , TJ (figure 6.13), capable of moving at an average speed of a little under three feet per second in an indoor office environment . The symbolic level handles where-to- go- next decisions (strategic) , whereas the subsumption level handles where-to- go- now choices (tactical ) . A coarse geometric map of the world is present at the strategic level , and route planning is conducted within this representation . Piecewise segmentation of the route in a manner similar to that of AuRA provides the behavioral configurationfor eachleg of theoveralljourney.

Also worth mentioningis an earlier system(Soldo 1990 consider ), ably less the , developedthat advocates useof behavioralexpertscoordinatedby an AI plannerusing a world map. This systemprovidesa frameworkfor integrating deliberativeplanning and reactivecontrol by allowing the plannerto choose behaviorsalongthe samelines asboth SSSandAuRA. appropriate

230

Chapter6

Figure 6.13 TJ. (Photograph courtesyof Jon Connell.)

. Multi -Valued Logic (Saffiotti et al. 1995 : Researchers SRI International at ) them formally, , drawingheavily on manyearlier ideasand synthesizing have developeda novel hybrid architecturethat uses a multivalued logic for ) (MVL ) representation behaviors(motor schemasas the reactivecomponent ' coupled with gradient fields as goals in the mannerof Paytons work (section 5.2.2). Multivalued logic provides the ability to have a variable . , plannercontroller interfacethat is stronglycontextdependentIn other words the decisionwhen to plan and when to react reflectsthe natureof the environment . Further behavioralplans are included that draw inspiration from , -style deliberation Theseprovide a form of preplanned behaviorthat can . PRS

"

231

/ Architectures Hybrid Deliberative Reactive

be invoked and elaboratedas necessaryThis systemhas been success . fully testedon the robot Flakey(figure 6.12) in variousindoor office environments . . SOMASSHybrid AssemblySystem(Malcolm andSmithers1990 : The SO) MASS systemis an assemblysystemconsistingof two parts the cognitive : . (deliberative and the subcognitive(reactive componentsThe cognitivepart ) ) consists a symbolicplannerdesigned be asignorantaspossible- a virtue of to . accordingto the systemdesignersThe intent is to avoidcloggingthe reasoner with unnecessary . knowledge The planner itself is hierarchicalin structure , concerned with finding a suitableorderingof partsto producethe required first with , assemblythensubsequently determining gravitationalstability consistent with the ordering producingsuitablegraspsto acquirethe part, insuring that , tolerances handlingerrors are met, and translatingthe plan into for assembly executable robot code consistingof parameterized behavioralmodulessuitable for execution The subcognitive . is concerned with the actual component executionof the behaviorsafter the plan is downloaded the robot. In this to on , system implemented a working robotic arm, thereis a clear cut division between deliberationandplanningbut a limited ability to exchange information between them. . Agent Architecture(HayesRoth et at. 1993 : Plansin this architecture are ) considered of intendedcourses behavior 1 levelsare speci of . \\10 descriptions fied within the agentarchitecturethephysicallevel, concerned : with perception and action within the environment and the cognitive level, for higher level , needssuchas problem solving and planning According to the designers . reasoning , finer resolutioncould yield more than two levels but currently this , numberseems . from the cognitive level to adequateA plan is communicated the physicallevel, with feedback from the execution the plan returnedto the of claim that reactiveand deliberative(planning cognitive level. The designers ) behaviorscan coexist within eachlevel, so the standard partitioning of reactivity anddeliberationdoesnot pertain As examplessituationassessment . can , occur within the cognitive level, and limited path planning can occur within the physicallevel. The differencebetween levelsis essentially epistemological andtemporal basedon the following distinctions : , . Whether symbolic reasoning(cognitive versus metric ) ) (physical is employed . . Time horizonsfor history and reactionare both significantly shorterfor the level compared the cognitiveone. to physical . Greaterabstraction presentat the cognitivelevel. is The system beentestedon mobilerobotsfor both surveil has lanceanddelivery tasks .

232

Chapter6

" . Theo Agent (Mitchell 1990 : " Reactswhen it can, plans when it must is ) at the motto for Thea Agent. This hybrid system developed CarnegieMellon , : , , University focusespredominantlyon learning in particular learninghow. to about becomemore reactive more correctin its actions and more perceptive , , . the world' s featuresrelevantto its performance Stimulus response(behavioral ) rule selectionand executionare the basisfor reactiveaction, using an rule. If no rules apply arbitrationmechanism choosethe most appropriate to , a then andonly then is the plannerinvokedto determine suitablecourseof action rule . As a resultof theplanningprocessa new stimulusresponse is added , . or to the existingrule set This newrule canbe usedagainlater shouldthe same -Agentwas . similar situationarise this time without the needfor planning Theo , cans A . testedon a Hero 2000 mobile robot given the task of locatinggarbage stimulusresponse reactiontook on the order of 10 milliseconds whereasthe , . plannerrequired severalminutes Hence if the robot begins with few or no from initially severalminutesto undera second rules its reactiontime decreases , more and more of the ) (about two ordersof magnitude as it experiences world. . GenericRobot Architecture(Noreils and Chatila 1995 : This hybrid architecture ) in , developed France consistsof threelevelsthat bridge the spectrum , of planningto reactivity: ' . Planninglevel: Generates the of sequences tasksto achieve robot s high-level ) goalsusinga STRIPSlike planningsystem(Nilsson 1980 . . Control systemlevel: Translates plan into a set of tasksand configures the . the functional level prior to execution . Functionallevel: Corresponds a setof functionalmodules(servoprocess es to . with reactiveexecution Implementedbehaviors to behaviors concerned ) analogous includeobstacleavoidancewall following, and visual tracking. , of This systemis similar in structureto several the selectionstyle architectures in . already encountered this chapter A significant contribution of this work that providesa formal of lies in the development a task descriptionlanguage . methodfor designingandinterfacingthesemodules Specialattentionhasalso . beenpaid to diagnosticand error recoveryproceduresThe systemhas been . testedon the Hilare seriesof robots using vision for trackingtasks , . DynamicalSystems has and ) (Schoner Dose 1992 : This approach Approach been significantly influencedby biological systemsresearchas a basis for . providing an integrativehybrid approachfor reactingand planning Suitable vector fields, designedusing potentialfield methods(section3.3.2), serveas the basisfor planning and provide a clean bridge to behaviorbasedreactive ' execution The deliberativeplanner operateswithin the reactivecontroller s .

233

/ Architectures Hybrid Deliberative Reactive

' , representational space dealing with the underlying conttoller s mathematics and dynamicsratherthan reasoning . symbolically Planningconsistsof selecting and providing parameters eachof the associated for behavioralfields and ' in detenniningtheir relativesttengthfor summation purposes light of the task s consttaintsThis work hasbeendemonstrated simulationonly to date but it . in , in which providesan interestingway of rethinking the representational space . high-level deliberativeplanningcanoperate . Supervenience Architecture(Spector1992 : The supervenience architecture ) for on providesan environment integratingreactionanddeliberationbased abstraction , in particularthe " distancefrom the world" (superveniencethat the ) abstracted . Althoughtheramificationsof this work areoften concept represents more concernedwith philosophythan robotics a multilevel implementation , of the architecturereferredto as the abstractionpartitionedevaluator(APE) hasbeenimplementedIt consistsof multiple levelsrangingfrom the perceptual . /manual(lowestlevel), to spatial temporal causal andfinally conventional , , , in . (highestlevel), connected a strict hierarchy To test theseideas a simulated , homebotcapableof actionssuchasgrab, move object, move right, rotate and , the like hasbeenused The main premiseis that reactivity anddeliberationare . differentiatedprimarily by their levelsof abstraction how far they are removed and from the real world. Supervenience providesan integratedformalism for describingthesemanylevelsof absttaction As such it somewhat . blurs the , distinctionsthat otherhybrid architectures makeandthusleanstowardsa more traditionalhierarchicaldesign(e.g., Albus 1991 Meyste I1986 ). , . Teleoreactive Architecture(Bensonand Nilsson 1995 : This hybrid Agent ) deliberativereactivearchitecture basedon the constructionof a plan in the / is form ofa setofteleoreactive(TR) operators section3.3.1) which an arbittator ( then selectsfor reactiveexecution The deliberativecomponent . involveshierarchicalplanning yielding a tree like structurethat consistsof TR programs , . The TR formalismprovidesthe unifying representation both reasoning for and . . reacting The systemhasbeentestedin a simulatedbotworld environment . ReactiveDeliberation(Sahota1993 : The reactivedeliberationarchitecture ) consistsof two distinct layers the deliberatorand the reactiveexecutor The : . executor consists actionschemas of at a level similar to that of RAPs operating (section3.1.3). The deliberatorenablesa single action schemaat a time and . refersnot to higher level givesit parametersDeliberationin this architecture abstractreasoningbut rather to the selectionof one of the many potential behaviorscurrently appropriate executionin the given situationconsistent for ' with the agents goals In manyrespects is merelyan elaborated . this versionof an action selectionmechanismbut it providesus with anotherway to think ,

234

Chapter6

. about the interface betweenplanning and reactivity This systemhas been . testedusing small robots for playing tabletop soccer The behaviorsfor this domainconsistof activitiessuchas shoot defendfed-line, clear go-to-home , , line, and so on. . Integrated and PathPlani1ing Dynamic SteeringControl (Krogh andThorpe in 1986 : In early work demonstrated simulation a strategyfor using path , ) world modelwascoupled methods relaxationovera grid-based using planning . with a potential fields- basedsteeringcontroller The path plannergenerated . controller Potentialfields referredto ascritical pointsfor the steering subgoals -time feedback similar to those methods ), (Krogh 1984 modifiedto providereal usedin AuRA, provided the local navigationalcapabilitiesfor achievingthe . the seriesof subgoals path plannerestablishedThoughnot really abehavior a clear integration betweenpath basedmodel this early exampleprovides , planningandreactivecontrol. . UUV s: Hybrid architectures have been applied to underseanavigational . tasks by severalresearchersThe rational behavioral model ( Byrneset al. 1996 a three layer architecture , ), consistingof execution tactical, and strategic . over behaviorsat different levels Although castin a more , layers reasons hierarchicalframework (cf. Saridis 1983 it usesprimitive behaviorsas the ), / 1991 usesGapps Rex . for planning Another system( Bonasso ) primary object and section3.2.4.2) asthe underlyingreactivecontrol methodology subsumption ( . ) (behaviors as the primary operators This is an example competences than of deliberationand more of hybridizing two different reactivestrategies robot. is reactivity. The targetvehiclefor both architectures an undersea

SUMMARY 6.7 CHAPTER

. Both deliberative have and planningsystems purely reactivecontrol systems in limitations when eachis considered isolation. . Deliberative planning systemsprovide an entry point for the use of traditional AI methodsand symbolic representational knowledgein a reactive . robotic architecture . The interfacebetweendeliberationand reactivity is poorly understood and . in asthe focusof research this area serves . Strongevidenceexiststhat hybrid deliberativeand behaviorbasedsystems arefound in biology, implying that they arecompatible symbiotic andpotentially , , suitablefor usein robotic control. . Hybrid modelsinclude hierarchicalintegration planningto guide reaction , , . andcoupledplanningandreacting

235

Hybrid Deliberative / ReactiveArchitectures . Another important design issue concerns the number of layers present within the overall architecture , with two or three currently being the most common . . AuRA is an early hybrid deliberative / reactive system using motor schemas AI spatial planner . The planner system prior to execution and reconfigures it in . Atlantis is a three - layer hybrid architecture introduced the concept of cognizant failure in and a traditional configures the reactive control the event of task failure . based on RAPs and NATs . It which a robot becomes aware

of its inability to complete a task . Plans are viewed as advice rather than commands or instructions in this system . . The Planner - Reactor architecture consists of two major components . Planning is viewed as continuous adaptation of the reactive component . The RS ' model provides the underpinnings of all the architecture s components . Situations , provide the context for sets of reactive actions . . PRS uses a least - commitment strategy to delay the elaboration of plans for execution until necessary . Although not strictly behavior based , it does react to changes in the environment detected via sensing and develops plans consistent ' with the robot s current observations , beliefs , desires , and intentions . . Selection ( AuRA ), advising ( Atlantis ) , adaptation ( Planner - Reactor ), and postponing ( PRS ) are four major interface strategies frequently used in various hybrid architectures . . Many other hybrid architectures have also been developed along similar lines : SSS , MVL , agent architecture , Theo - Agent , Supervenience , and teleo reactive agent architecture among others .

Chapter Perceptual

7 Basis for Behavior - Based Control

' We don t see things as diey are, we see d1emas we are. ---.Anais Nin

" " It would be as useless to perceive how things actually look as it would be to watch die random dots on untuned television screens. - Marvin Minsky

We have to remember that what we observe is not nature in itself but nature exposed to our med1od of questioning . - Werner Karl Heisenberg

Obje Chap an mo be bet intim the pe 1utilit .unde To,re rela rob info can how pr 2biol de appr .exp ofthe perc .as mo the Tode 3beh .expl str per reco att foc role 4offo ,of, pe -pe .se b th with fusio sens and o see seve 5 exa repr . cl robo syst

238

7 Chapter

7.1 A BREAK FROM TRADITION

Without a doubt a robot' s ability to interpretinformation aboutits immediate , is crucial t~ the successful achievement its behavioralgoals of . surroundings To react to externalevents it is necessary perceivethem. The real world to , is often quite hostile to robotic systems Things move and changewithout . , , warning at best only partial knowledgeof the world is available and any a informationavailablemay be incorrect inaccurateor obsolete . , , priori Machineperceptionresearchand in particularcomputervision, hasa long , and rich tradition, with a great part of it dissociated from the issuesof realtime control of a robotic system This work hasfocusedon taking input sensor . andproducinga meaningfulandcoherentsymbolicand or geometric / readings of the world. The top panel of figure 7.1 represents viewpoint this interpretation . Much of this researchhas ignored the fact that perceptualneedsare ' predicatedupon the consumingagents motivationaland behavioralrequirements . can in Certainlyscapegoats be found for the lack of progress producingrealtime robotic perceptionunderthis paradigm Computerarchitectures : weretoo havenot provided an adequate of primitive, or neuroscientists understanding humanvision. Perhaps however the meanswere not at fault, but rather the , , desiredends The traditional approach significantproblems . has : . Perceptionconsidered isolation: Is it wise to considerthe perceiveras a in disembodied ? This is perhaps similar to studyinga living creatureby process it up andhandingout the piecesto different scientists Perception . is chopping betterconsidered a holistic, synergisticprocess as deeplyintertwinedwith the ' . completeagents cognitiveandlocomotionsystems . Perceptionas king: There hasbeensomeelitism regardingmuch of the research in perceptual . , processingcomputervision in particular Unquestion , ably vision is a hardproblem Nonetheless . activitiesneedto be viewedas perceptual for . only oneof the manyrequisiteneeds a functioningintelligent agent Vision researchers benefitgreatly by considering can theseother systemcomponents
as partners, as opposed to servants. . The universal reconstruction : Much perceptual research has focused on creating three-dimensional world models. These models are often built without ' regard for the robot s needs. A deeper question is whether these reconstructive models are really needed at all . Roboticists (Brooks 1991b) and psychologists (Neisser 1993) alike lament the pitfalls associated with the traditional approach to machine vision . Over-

239

J PerceptuaBasisfor BehaviorBasedControl

Figure7.1 of . (Figure es ) courtesy BobBolles. Approach to perception

"

PARADIGM

PARADIGM det

NEW

OLD

240

Chapter? coming these difficulties requires a shift toward a new (or rather rediscovered) paradigm : viewing perception as a partner process with action. More accurately , a duality exists: The needs of motor control provide context for perceptual processing, whereas perceptual processing is simplified through the constraints of motor action. In either case, action and perception are inseparable. Recently, in developments paralleling the advent of behavior-based robotic systems, new approaches have emerged that take this interplay into account. These methods are guided by the adage: without the contextof actionis meaningless . Perception The reflections of the new perceptual paradigm include : . Action - oriented perception : An agent' s perceptual processing is tuned to meet its motor activities ' needs. . Expectation -based perception : Knowledge of the world can constrain the interpretation of what is present in the world . . Focus-oj -attention methods: Knowledge can constrain where things may appear within the world . . Active perception : The agent can use motor control to enhance perceptual processing by positioning sensors in more opportune vantage points . . Perceptual classes: These partition the world into various categories of potential interaction . The bottom panel of figure 7.1 captures some aspects of this new approach. Perception now produces motor control outputs, not representations. Multiple ' parallel processes that fit the robot s different behavioral needs are used. Highly specialized perceptual algorithms extract the necessaryinformation and no more: Perception is thus conducted on a need- to -know basis. To further advance this position , complexity analysis of the general task of visual search has provided illuminating results. Bottom - up visual search where matching is entirely data driven has been shown to be NP - complete and thus computationally intractable , whereas task- directed visual search has linear - time complexity ( Tsotsos 1989) . This tractability results from optimizing the available resources dedicated to perceptual processing (Tsotsos 1990) . Attentional mechanisms that result from exploitation of the knowledge of the specific task provide just such constraints. The significance of these results for behavior-based robotic systems cannot be underestimated: " to Any behavioristapproach vision or roboticsmust deal with the inherentcomputa tional complexityof the perception : es problem otherwisethe claim that thoseapproach scaleup to humanlike behavioris easily refuted (Tsotsos1992 p. 140 ." , )

241

Basisfor BehaviprBasedControl Perceptual The net outcome is that a primary purpose of perceptual algorithms is to support particular behavioral needs. In earlier chapters, we have seen that behaviors and their attendant perceptual processes can be executed in parallel . In reactive control , sensor information is not fused into a single global representation over which other planning processes then reason. This is in marked contrast to more traditional hierarchical views of robotic control which assume that perception ' s purpose is to construct a global world model (Barber a et ale 1984) . The inherent parallelism and more targeted processing of behavior based robotics permits much more efficient sensor processing. To emphasize further the importance of perception itself , we revisit the symbol grounding problem in AI (chapter 1) . Perception provides perhaps the only opportunity for us to provide physical grounding for the objects within ' ' .an agent s world . The agent s interaction with these objects completes the grounding process by providing meaning through its resulting actions.

7.2 WHATDOES BIOLOGY ? SAY

A wide rangeof disciplineswithin the biological sciences haveaddressed the issuesof perceptionasrelatedto behavior For roboticists significantinsights . , can be gleanedfrom thesestudies This section provides an overview of a . few importantresultsfrom research perception neurosciencepsychology in , , , and ethology of particular relevanceto our study of behaviorbasedrobotic . systems

7.2.1 TheNatureof Perceptual Stimuli

To begin with, it is useful to distinguishbetweenthe different ways of categorizing perceptualstimuli. One such distinction can be basedupon the of the received stimuli. Proprioception refers to perception associated origin with stimuli arising from within the agent This includes information . such as tendon or muscle tension from which limb position or the number , of times a particular action has beenrepeated(such as a leg movement ) . with external might be computed Exteroceptionrefers to perceptionassociated stimuli. Here the environmenttransmitsinformation to the agent via vision, audition or some other sensormodality. The most common industrial , robotic arms that computetheir end effector positions through inverse kinematicsrely on proprioceptive information. If , however a vision systemis , to the robot, exteroceptive data can provide environmental feedback coupled asto wherewithin the world the robot needsto move Clearly reactiverobots . ,

242

Chapter7

behaviorsrely heavily tightly coupledto the environment throughsensorimotor on exteroceptive . control hasbeen , perception Nonethelessproprioceptive ' of widely observed biologistsin animals generation navigationaltrajectories by . One of many suchexamples occursin insectssuchas millipedes(Burger and Mittelstaedt 1972 and in desertspiders( Mittelstaedt1985 wherehoming ) ), behavioris basedupon proprioceptivesensations is generated and from " the " sum of the momentaryperipheralafferent inputs ( Burgerand Mittel staedt1972 . In other words the distances traveledby the insec and spiders , ) ~ are believedto be storedin somemanner then used later by the organisms , to return to their homesites This process referred to as path integration . , , relies entirely on proprioceptiveinputs. The sensorydata generated an , by accumulationof the animal' s past movements and usedto orient the animal within the world, is referredto as ideotheticinformation ( Mittelstaedt1983 . ) In contrast orientation information generatedby landmarks sun position , , , or other externalcues is referredto as allothetic information. Allothetic information supportsclosedloop control basedon continuousfeedbackfrom an external source whereasideotheticinformation providesonly open loop , control and is thus subjectto greatererror due to the inevitablenoise during locomotion .

In animal navigation , it is believed that both allothetic and ideothetic information are in use and integrative mechanisms are provided to reconcile the inevitable differences between them smoothly . We will study the issues concerning the combination of multiple , potentially conflicting data sources in section 7.5.7. 7.2.2 Neuroscientific Evidence

the

and

to

output

do

between

its to

inputs

of available is the of difference

results the commanded one

the feedback as was no , it

from is used evaluate is to that what ,

feedback means result feedback no

between

uses has This actual . the system system deviation and the

control control action accomplished loop loop compute . to actually open it

closed commanded An A

actions

what

the

controller

. .

243

Basisfor BehaviorBasedControl Perceptual

informationis processedUnfortunately we do not know asmuchaswe would . , like about the actual processingof perceptualinformation within the brain. Nonethelessseveralrelevantobservations derivedfrom neuroscience , may be ' s role in behavior in our understanding perceptual of . helpful processing Individual sensormodalities have spatially separated regions within the brain. Sight, hearing and touch all have distinct processingregions Even . , within a specificsensor is :" the type, spatialsegregation present Perhaps most striking finding is that thereis no single visual areain the brain. Different areas of the brain specialize different aspects vision suchasthe detectionof in of color, movement and intensity . . . " (McFarland1981 p. 593). This , , pattern , observation holds not only for the humanbrain but that of lower animalsas well. For example a distinct neuralregionexistsfor loomingdetectionin frogs , -Perez1995 . Neural structures associated with predatoravoidance Cervantes ( ) associated with prey selectionfor theseanimalshavealsobeenobserved Fite ( 1976. ) In the humanandprimatebrain, visual processing channeled two distinct is into vision streams( Nelson1995 : the object vision stream concerned with ) , of objectsand foregroundbackgroundseparationand the spatial recognition , vision stream which providespositional information useful for locomotion . , " Theinitial evidence these" what" and" where visual systems for camefrom lesion studiesconducted primates( Mishkin Ungergleiderand Macko 1983 on , , ) that the object streamis localizedto the temporalareaof the cortex indicating , whereas parietalregionsareassociated the with spatialvision. Furtherspecialization occurswithin the cortexitself. Orientationsensitivity to a particularstimulusoccursthroughoutlayersof the visual cortex A neuron . at a particularlevel is sensitiveto a stimulusat a preferredorientation as has , beenobserved catsand macaque in , ) monkeys( Lund Wu, andLevitt 1995 . Echolocationanalogous sonarsensing robots alsohasspecialized to in neural , , with , regionsassociated it. In particular the auditorycortexof the mustached bat is dedicated this type of processing hasadditionalparce to but Uationwithin itself (Suga and Kanwal 1995 . The subdivisionsare associated with varying ) , rangesto targets eachof which likely hasa differing behavioralresponse associated with it. Further analysishas revealeddifferent specialized regions associated with targetsizeandvelocity. One final observationthat we mention is the spacepreservingnature of the connections betweenthe brain and the sensingsystemitself. Thesemap pings areprevalentand areexemplifiedby the retinotopicmapsprojectingthe ' eyes output throughthe lateral geniculatenucleusonto the visual cortex; somatotopicmapsprojecting the peripheralinputs generated touch onto its by

244

Chapter?

associated cortical regions and tonotopicmapsfound preservingspatialrelations ; audition. Sensoryinformation impingesupon the brain in a producedby mannersimilar to its externalsource .

7.2.3 Psychological Insights

The observerwhen he seems himself to be observinga stone is really, if physicsis to , , to be believed observingthe effectsof the stoneupon himself. , - BertrandRussell Finally , taking a psychological perspective, we can obtain additional insight . In particular we draw heavily upon the theories of J. J. Gibson and Ulric Neisser ' regarding perception s role in generating behavior.

7.2.3 .1
A relevantand important conceptlies in the meaningof objects in relation ' to an organisms motor intents a conceptGibson ( 1979 first introducedas , ) " . , affordancesAs definedby Gardner( 1985 p. 310), Affordancesare the potentialities for action inherentin an object or scene the activities that can an take place when an organismof a certain sort encounters entity of acertain " The Gibsonian of affordances formulatesperceptual entities sort. concept but not as semanticabstractions ratherby what opportunitiesthe environment affords The relationshipbetweenan agent and its environmentafforded by . . a potential action is termed an affordance All information neededfor the are to act resideswithin the environment and mental representations , agent . not usedto codify perception A chair can be perceiveddifferently at different times, as somethinguseful to sit in, as somethingblocking the way, or . as somethingto throw if attacked The way the environmentis perceiveddepends on what we intendto do, not on somearbitrary semanticlabeling (e.g., chair). A chair need not be explicitly recognizedas a chair if it is serving , only as a barrier to motion. Underthosecircumstancesit needbe recognized as an obstacle If tired, it need be recognizedonly as a place to rest. . only ' into designingalgorithms this From a robot designers perspective translates that impede motion, afford rest or protection or other capabilities that detectthings , but not in to designingalgorithmsthat do semanticlabeling and . categorization affordance research Adolph, Gibson and The following stepscharacterize , ( : ) Eppler 1990

245

Basisfor BehaviorBasedControl

I . Describethe fit betweenthe agentandits environment . 2. Detenninethe agentenvironment relationships regardingboth the optimal of the action andthe transitionsbetweenactions . performance 3. Analyze the correspondence between the actual and perceived agent environment fit. 4. Detenninethe perceptual informationrequiredto specifythe affordance . 5. Evaluatehow to maintainand adaptaction asnecessary . Theseguidelines developedfor psychologists can also be of benefit in the , , of perceptualalgorithms to support behaviorbasedrobotic systems . design ' of the more radical camp that strictly maintain Gibsons views have Many , lately fallen on hard times within the mainstream psychologicalcommunity but that in no way diminishes the potential value of his ecological stance . on agentenvironmentinteractionsas a basis for robot perceptualalgorithm , ; ; generation(Pahlavan Uhlin and Ekhlundh 1993 Blake 1993 Ballard and Brown 1993 Arkin 1990a. We, as roboticists will use the term affordance ; , ) to denotea perceptualstrategyusedto interactwith the world, satisfyingthe needof somespecificmotor action.

7.2.3.2 A ModifiedAction-Perception Cycle

While I am not sure that access movementproducedinformation and affordances to would be sufficient to produceperceptualawareness a machine it is a necessary in , condition . . . (Neisser1993 p. 29) ,

' Neisser( 1989 modifies the Gibsonianstancesomewhatto permit Gibsons ) to ) ecologicalperspective accountfor the spatialvision stream(for locomotion discussed earlier in this section while forwarding a cognitive explanationfor , the object vision stream(for recognition. This approach recent ) acknowledges a of neurophysiological findings and presents two-prongedexplanation vision reason with the approach usedin someof the hybrid robotic es ably consistent architectureswe encountered chapter 6. Other robotics researchers in have theseparallel pathwaysand used them to constructseparate recognized yet coordinated vision systems determiningwhat an objectis apartfrom where for it is located (Kelly and Levine 1995 . If indeed there are multiple parallel ) in thebrain asthe evidence indicates it is certainlypossible , perceptual systems that different methods exist aswell of processing informationfor action. that ' Neissers perspective arisesfrom the schoolof cognitivepsychology(chapter 2) and leadsus to the notion of action-orientedperception This school .

246

7 Chapter

....- -----......, " // / ~ ACTION PLANS WORLD BEHAVIORS ~'""..reactive""/, ' ""---------., "" shunt
direction

ODELS MEMOR
7.2 Figure Modified action perceptioncycle.

cognitiol1

of thought acknowledges fact that perceptionand action are intimately the intertwined Neisser( 1976 elaborates action perceptioncycle asthe basis . the ) a ( by which humansinteractwith their environment figure 7.2 presents modified version . In this cycle, perceptions from interactionwith the world ) arising ' and , modify the organisms internal expectations behaviors which in turn result in new exploratoryactivities that result in new perceptionsAnticipatory . schemas playa crucial role in providing both the direction and contextfor interaction ' with the world. Neissers initial versionof this cycle doesnot include the reactiveshunt which, in my estimation more directly ties perceptionand , , of . action togetherwhile still permitting the coexistence plansfor actions The with his later publications versionpresented hereis believedto be moreconsistent 1989 which haveevolvedsincethe earlierexpositionin Neisser ( Neisser ) . 1976 In any case this liberty was takento provide a betterreflectionon how , this cycle canbe relatedto behaviorbased robotic systems .
7.2.4 Perception as Communication - An Ethological Stance

Considerthat the world is trying to tell us something if only we knew how , to listen. Sensingcan thus be viewed as a form of communication in which . information flows from the environmentto the attendingagent Obviously if , we don' t know what to attendto we will have a hard if not an impossible , , that time discerningthe messages the world is providing. The world is telling . us somethingif only we would pay attention Whereand how our attentional and perceptualresources directeddependsstrongly on our motivation or are intentionalstate .

247

Basisfor BehaviorBasedControl Perceptual

The ethological literature is replete with examplesof sensed information ; ) providing cuesfor evoking behavior(e.g., Smith 1977 Tinbergen1953 . Indeed to , evolutionhasprovidedbiological agentswith highly tunedapparatae the information necessary carry out useful actions The to . pick up efficiently Ctors Ewert 1980 mentioned earlier for guiding visual ( ) looming andprey dete in . response the frog are goodexamples In recognitionbehavior we find that someagentsare capableof discerning , kin thingsotherssimply cannot(e.g., intraspecies recognitionamongbirds ' 1983 . Perceptual cuesnecessary an organisms survivalandroutine for (Colgan functioning are extractedcheaply and efficiently from the environment whereasirrelevantinformation is not processed all (i.e., it is not evendiscarded at : pick up neveroccurs . In otherwords theseagents haveevolvedmechanisms , ) that enableefficient communicationwith the world' s salient features ' , (salient that is, in the contextof that agents needs. This implies that we need ) to haveour robotic agentsattendto what is necessary the context of their in . , (not our) needs Dependingon their internal conditions motivationalstateor and sensorylimitations, we can developalgorithmsthat provide useful , goals andfocusedinformationfor theseactors .

7.3 A BRIEF SURVEY ROBOTIC OF SENSORS

Sensortechnologyhasadvanced , rapidly in the last decade resultingin many -costsensor low that robots . systems canbe readilydeployedon behaviorbased can Sensors be categorizedin termsof their interactionwith the environment , , as either passiveor active Passivesensorsuse energy naturally presentin . the environmentto obtain information. Computervision is perhapsthe most . typical form of passivesensing Passivityis particularly importantin military . , , applications wheredetectionof the robot shouldbe avoided Active sensors on theotherhand involvethe emission energyby a sensor of into the , apparatus environmentwhich is then reflectedback in somemannerto the robot. llitra , sonicsensing laserrangefinding aretwo commonactivesensor and modalities usedfor behaviorbased robots . A very brief discussion the operationof several of sensor systems representative follows, including the use of shaft encoders deadreckoning Shaft for . encoders not environmental are sensors the strictestsense sincethey measure in , ' only the rotationsof the robot s motors(i .e., they provide proprioceptive information . Nonethelessthey are widely usedfor positionalestimationand , ) . in warrantfurther discussionThe readerinterested moredetailedinfonnation . on a wide rangeof sensors usefulfor robotsis referredto Everett 1995

248

7 Chapter

7.3 DeadReckoning .1
Dead reckoning(derivedoriginally from deducedreckoning providesinformation ) regardinghow far a vehicle is thought to have traveledbasedon the rotation of its motors wheels or tteads(odometry. It doesnot rely on environmental , , ) .1 \\10 generalmethodsfor deadreckoningare available : sensing shaftencoders inertial navigationalsystems and . Shaftencoders by far the mostfrequentlyusedmethodof deadreckoning are because their low cost. Theseoperate maintaininga countof the number of by of rotationsof the steeringanddrive motor shafts(or wheelaxles andconverting ) thesedatainto the distancetraveledand the robot' s orientation Although . shaft encoders can provide highly reliable positional information for robotic arms which are fixed relative to the environment through direct kinematics , , , in mobile systems . they canbe exttemelymisleading

Direct the of robot the

kinematics ' s end robot ann

involves effector .

solving is , given the

the joint

necessary coordinates

equations ( angles

to and

detennine length of

where links )

If you haveeverbeenstuck in ~ ud or snowin an automobile you can recognize , that the informationasto how manytimesthe drive wheelshaveturned does not necessarilycorrelatewell with the car' s actual positional changes . Becauseshaft encodersare proprioceptiveand only measurechangesin the robot' s internal state they must be supplemented with environmental , sensing to producereliable resultswhendeterminingthe robot' s actuallocationwithin the world. An inertial navigational system (INS) does not measurethe rotation of wheelsor shaftsbut rather tracks the accelerations robot has undergone the , this information into positional displacementsThis resultsin far . converting more accuratedead reckoning systems but with one major penalty higher : , cost. INS is also proneto internal drift problemsand must be periodically recalibrated to yield accurateinformation. The quality of the data makesit far moredesirable thanshaftencodersbut costandpowerrequirements , frequently its deployment . prevent , Although not basedon deadreckoning global positioning systems(GPSs ) can also provide geographic information as to the robot' s whereabouts within

249

Basisfor BehaviorBasedControl Perceptual

the world. A battery of twenty-four Departmentof Defenseearth orbiting satellitesrelay positional data wherebythe robot can deduceits position relative to a .world coordinatesystem Time of flight of the GPS signals and . with threetransmittingsatellitesenables robot' s altitude longitude the , triangulation . , and latitude to be computed Global positioning systemsare rapidly ' in decreasing cost but cannot be used inside buildings where the satellites . ) signalsare blocked Differential GPS (DGPS can provide higher positional resolutionthan the standard GPS service which is limited to approximately , nonmilitary lOOmeter accuracy(bestcase (Everett 1995 because an intentionally of ) ) use degraded public usagesignalpreventingunintended by hostile . the military powers DGPSrequiresa ground basedtransmitterto supplement satellitesandcaneasily yield relativeaccuracies submeter in . ranges

7.3.2 mtrasound
Sonar(ultrasonicsensing is a form of activesensing It operates the same . on ) basic principle by which bats navigatethrough their environment A high. click of soundis emittedthatreflectsoff a nearbysurfaceandreturns frequency later at a measurable time. The delay time for receipt of the returning signal can be usedto computethe distanceto the surfacethat reflectedthe soundif the velocity of the soundwaveis known. A typical ultrasonicsensor Polaroid ) ( emitsa beamthatreceives echoes from a regionapproximately degrees wide 30 . can operaterapidly, returning ten emanatingfrom its source Thesesensors or more depthdatapoints per second Accuracyfor many working systems . is on the order of centimeters(0.1 foot) over a maximum rangeup typically tometers . A wide rangeof sensors commercially availablecovering a is . broadrangeof frequencieseachwith variationsin beamwidth and distance , sonarring equippedwith sixteen 7.3 showsa Nomadic Technologies Figure sensors . : It illtrasonic sensing decidedadvantages is of low cost, providescoarse has -dimensionalenvironmental information (distanceto an object , and returns three ) a tractableamountof data for interpretation Its disadvantages substantial . are aswell: It hasmuchpoorerdiscriminatoryability thanvision, is significantly conditions frequently to , susceptible noiseanddistortiondueto environmental erroneous databecause reflectionsof the outgoingsound of produces . waves and the sonarbeamis proneto spread Sonarhasfound its bestusein , at . obstacle detectionandavoidance shortrange Its difficulty in discriminating different typesof objects for example betweenan obstacleand a goal, limits , ,

250

Chapter7

Figure7.3 Ultrasonic sensors photograph .( of Inc. courtesy Nomadic Technologies , Mountain View California .) ,
its applicability . Sonar cannot be used in outer space as it requires a medium , for the transmission of the sound wave. Progress has been made in the use of phased sonar arrays to provide greater information regarding the environment , but these currently are not in widespread use in robotic systems. 7. 3. 3 Computer VISion

Video technologyhasbeenavailablefor at leasthalf a century Only recently . , however has chargecoupled device (CCD) camera technology advanced , rapidly in terms of miniaturizationand greatly lowered cost. Color imagery is now availableat very affordableprices . , , Although somedigital cameras are available most robot vision systems consistof one or two black-and white or color analogoutput CCD cameras , oneor moredigitizers, andan imageprocessorBecause . robotsneedreal-time architectures , interpretationof incoming video data they often requirespecialized found in image processors seriousresearchand applications Cerfor .

251

Perceptual Basis for Behavior -Based Control

tain techniques , , provided throughthe useof behaviorbaseddesign however -cost completevision systems robots ( Horswill and can provide very low for , Yamamoto1994 . Figure 7.4 showsa high-end vision systemusedfor real) time tracking of peoplein an autonomous . helicopter It consistsof a low-light cameraor conventionalVideomountedon a pan tilt systemwith specialized image stabilization and real-time motion detection hardware(Cardozeand Arkin 1995 . A behaviorbasedhelicopter designedby Montgomery Fagg ) , , , and Bekey ( 1995 at the University of SouthernCalifornia won the 1994International ) , Aerial RoboticsCompetition This vehicle integratedthree sonar . sensors altitude measurements compass headingcontrol, three gyroscopes for for ,a for controlling attitude and a video camerafor recognizingtarget , . objects The sheervolume of video information generated be staggeringFor a can . camera typical imageresolutionafter digitization is on the orderof 512 , single ) by 512 pixels (pictureelements, with eachpixel consistingof eight bits of information . encoding256 intensitylevels Multiply this valueby threefor color one image plane eachfor red, green and blue) and then attemptto , images( at frame rate (30 times per second. We now havea receivingbandwidth ) process of approximately megabytes second Specialized often costly 24 ! and per . , image processinghardwarecan make this data flow tractable We will see however that behaviorbasedrobotic perceptionprovidestechniquessuchas , , the useof expectations focus of-attentionmechanismsthat constrainthe and , amountof raw data that must be analyzed significantly reducingthe overall , . processing requirementsThesebehaviorbased perceptual algorithmsaregenerally to exploit task and behavioralknowledgewhereverpossible . designed that track featuresover multiple frames are also commonly Adaptive techniques used Full-scalesceneinterpretation the hallmark goal of mainstream . , -understanding researchis generallynot required . , image
7. 3.4 Laser Scanners

Laser scanners active sensorsemitting a low-poweredlaser beamthat is are , scanned over a surface Throughtechniques . suchasphaseamplitudemodulation , the distanceto the individual points can be computedwith the net result an array of image points, eachof which has an associated . depth In effect, a -dimensionalimageis obtained Reflectance three . datais often also available , providing data regardingthe natureof the surfaceas well. The product is an datasource Figure7.5 illustratesa representative . extremelyrich three dimensional availablesystem . commercially

252

Chapter

(A)
I

(B) Figure7.4 . The Visionsystem anautonomous for (A helicopter helicopter ) hasa colorCCDvideo . to mounted a pantilt mechanism the nose The videois transmitted on on camera is . (Photograph theground station the (B), system where realtimetracking conducted .) of courtesy MarkGordon

Indy

Teleos

tracking

tlon

Video

I ~ I ~ I I I

host

SUN

r- - - - - - --I 1 MPC _. . . . . 1work 1 :.- - station Ethernet-- - - --: 1 .

253

Basisfor Behav -BasedControl :ior Perceptual

Figure 7.s LASARTM Laserscanning . ron system ( Photograph courtesyof Percept , Inc.)

Laser systemsare not without their drawbacks First and foremostis their . overultrasonicrangingdevices Anotherproblem . cost manyordersof magnitude , arisesfrom the mechanical instabilitiesassociated with manycurrentimplementations of thesedevices(e.g., problemswith nodding mirror designs usedfor mechanicallyscanningthe laserbeam . More subtledifficulties arise ) asa resultof the sparse datasamplingfound at longerdistancesa consequence ( of the imaging geometry which can causecertainobjectsto go undetected , ), and problemswith rangeperiodicitieswhen using phasemodulatedsystems that force interpretationambiguitieson the incoming dataregardingdistance to the surface Lower-cost linear array laser scanners available but they . are , consider less information. Nonethelessas the underlying sensor , provide ably

254

Chapter? technology improves in the next several years, these devices are expected to become more and more useful and commonplace . Their nature and use of low powered lasers nonetheless poses some difficulties for various applications in terms of both safety and stealth.

7.4 MODULAR PERCEPTION

A child of five would understand . Sendsomeone fetch a child of five. d1is to
- Groucho Marx

characteristics behaviorbasedroboticsis the designof of One of the essential motor behaviorsto provide overall control. This leavestwo multiple parallel choicesfor tying in perception generalized : . perceptionor modularperception ' ) RevisingMarr s ( 1982 definition of generalvision, generalized perception is a processthat creates given a set of input sensing a complete and accurate , , of the sceneand its properties Thus stated perception . , representation ' is conductedwithout regardfor the agents intentionsor availablerepertoire ? of behaviors But is therereally a needfor sucha scenerepresentationThe . cues roboticist, needs to identify the necessary behaviorbased only perceptual to supportthe needed motor actions Much of . within the environment required es the difficulty inherentin the generalperceptionproblemvanish , sincethere -fledgedscenereconstruction is no needto perform the complexand arduoustask of full . the As an alternative modular perceptionadvocates design of perceptual , dedicatedto extract the relevant information for each active behavior specialists . This reprisesthe themeheardin Minsky' s Societyof the Mind: " Each mental agentby itself can only do somesimple thing that needsno mind or thought at all. Yet we join theseagentsin societies in certain very special " - this leadsto true intelligence ( Minsky1986 p. 17)."We havealready , ways studiedthe behavioralagents , comprisingour robotic designs we now focuson . the individual moduleswhich, in toto, constituteperception Oncecommittedto the paradigmof modularperceptionwe mustdetermine , a . whatconstitutes module Variousdefinitionshavebeenforwarded which we , now review .

7.4.1 Perceptual Schemas

Schema theory has a long and rich history, which we reviewedin chapter 2. of We focus now on thoseaspects schema theory that apply to perceptionand . to can be generalized robotic systems According to Arbib ( 1995a p. 831), ,

255

Basisfor Behavi r-BasedControl Perceptual ~

"A es perceptualschemaembodiesprocess for recognizinga given domain of interaction with various parameters , , representing propertiessuch as size location and motion." Extracting severalof the major featuresof schema , ) theoryrelevantto modularperception(Arbib 1992 : . Schemas a set of multiple concurrentactive not passive process focused are es , , on differing perceptual activities . . Schemas contain both the control and knowledgerequired to completea task. perceptual . Schemas form an active network of process functioning in a distributed es mannerspecificfor a particularsituationand a setof agentintentions . . Schema thebasisfor languages defineaction orientedperception to theoryprovides (e.g., RS ( Lyonsand Arbib 1989 and the AbstractSchema ) Language .(Weitzenfeld1993 . . The activationlevel associated with a schema be relatedto the degree can of belief in a particularperceptual event . ' schemas somewhat are relatedto Gibsons affordancesserving , Perceptual a similar purposewithin an organism(Arbib 1981 Arkin 1990a. For the , ) roboticist the difficulty lies in how to operationalize notion of affordance this , asa schema . Each individual perceptualschemais createdto produceonly the information for . necessary the particular task at hand Rememberthat perceptual schemas embedded are within motor schemasproviding the information required , for themto computetheir reactionto the world (section4.4). The question becomes of saliency How do we know what featuresof the environment one : arethe correctonesto supporta particularbehavior Evenassuming ? that these beclearly identified we mustthenassess it is feasibleto extractthis can if , information in real time, as is necessary robotic control, given the limitations for of existing sensorandcomputational . technology Gibsonianaffordances are preoccupiedwith the role of optic flow in navigationaltasks Although . someprogresshas been made in using optic flow fields for behaviorbased control (Duchon Warren and Kaelbling 1995 often other more computa , , , ), -based tionally feasiblefeatureextractionalgorithmsare preferredfor schema . perception Let us examinesomepractical casesfrom our robot' s world. If an avoid static obstacle schema active asis usuallythecasesotherobot will not crash is , into things, some meansfor perceiving obstaclesis necessaryAn obstacle . is defined as somethingthat provides a barrier to the robot' s motion, that is, it occupiesspacein the intendeddirection of motion. Objects within the until they get in the way. Further environmentare not considered obstacles ,

256

Chapter
: , , they do not have to be semanticallylabeled as chairs people or whatever needto be recognizedas an impedimentto motion. The sensor they merely or algorithmicsourceof the perceptionof the obstacles of no concernto the is motor schemaonly the informationregardingwherethe obstacles located are , and if available a measure the certaintyof their perception It is irrelevant of . , , to the motor schema the obstacle if informationarisesfrom vision, ultrasound , or someother form of sensing From a designstandpoint it is necessary . , only to chooseone or more of thesesensoralgorithms embeddingit within the , motor schemaitself. The obstacledetection algorithm is unconcerned with other perceptualprocessing other active motor schemas thus can run for and within the context of the avoid static obstacleschema No . asynchronously world model of the environmentis required only reports as to where any , obstaclesare currently located relative to the robot. As this information is ' on the ( ), egocentric centered therobot s position it eliminates needfor absolute coordinateframesof reference . Anotherexampleinvolvesroadfollowing whenusinga stay on-pathschema Arkin ( 1990a. The questionof what perceptual featuresmakea road or path ) to a robot hasno single answer Roadsand pathsvary widely in . recognizable . , and appearance weatherandtime of day further alter their visual presentation ' For certainconditions a possiblesolutioninvolvestracking the paths boundaries , . One particular method usesa fast line-finding algorithm and is most successful whenappliedto well-definedroads paths or hallways(figure 7.6). , , An alternative for usein different situations exploitsa fast schema , , perceptual are region segmentation algorithm that is robustwhen the path boundaries ill defined(figure 7.7). In someinstances may be preferableto have both of it theseperceptualprocess active arbitrarily choosingthe most believableor es , fusing their resultsin somemeaningfulway (section7.5.7). Many other more , waysexist to perform a road following behavior someof which sophisticated in arediscussed section7.6.1.

7.4.2 VISUal Routines

Visual routines as developed Ullman ( 1985 are anothermethodfor describing , ), by modularperception In this approach a collection of elementalperceptual . , into a rich set of visual routines These . operationscan be assembled routinescanbe createdto servespecificperceptual ; they can sharecommon goals elementaloperationsand can be applied at different spatial locations within the image Controlmethods . mustbe providedfor sequencing routine the . correctly andapplyingthem at suitablelocationswithin the image operations

257

Basis for Beha~ior - Based Control Perceptual

Visual routines embody both sequential processing through the choice of an ordered set of elemental operations as well as spatial, temporal and functional (i .e., specialization ) parallelism . Base representations are first created to which the visual routines are then applied . This approach has a more bottom - up processing flavor than perceptual schemas since a continually available substrate of low -level perceptual ' processing is assumed to be available. This strategy is consistent with Marr s ( 1982) priffial and two -and- a-half- dimensional sketches. Box 7.1 describes these aspects of Marr ' s theory of vision .

Box7.1
: representation of levels three uses vision of theory ) 1982 ( s ' Marr ) The The The explicit at within . . .

In Ullman' s theory a setof universalroutinesprovidesinitial analysis any of , . the , providing image Theseuniversalroutinesbootstrap interpretation process indices and thus guidancein the application of more specializedroutines . There is inherently less relianceon expectations provide effective choice to in applying the set of perceptualmodules The applicationof visual routines . is not purely bottom up, however because routinesare assembled an as on , neededbasisfrom a finite set of elementaloperations Although a definitive . set of elementaloperations not beencreated severalplausibleoperations has , include . shifting the focusto different locationsin the baserepresentation . boundedactivation restrictingthe applicability over the spatialextentof the , baserepresentations . boundarytracing . location marking " . indexing basedupon a locations sufficient distinctiveness from its surroundings ,

explicit

shape

brightness ( s ' surfaces object

intensity an about . )

in regarding changes orientation information . makes surface regarding , . textures information . g . ) e ( and sketch , makes volume image , .

blobs

infonnation , g . the e model (

dimensional -

edges

makes as within half explicit a -

Sketch such point , dimensional and world each

the

Primal three two

258

Chapter?

(A ) Figure 7.6 Fast line finding. (A ) A Denning robot conductssidewalkfollowing using a fast line to finder The algorithmusesexpectations anticipatedie positionof die sidewalkboundaries . in S ) ( bod1 tenD of spatialand orientationconstraints. ( B) and (C) The resultsof . die groupingprocessand die computedcenterline Theseresultsare fed forward into . down the sidewalk as die next incomingframe asexpectations die robot proceeds

This theory intended primarily as an explanationof biological perception , es of , has led to the development severalapproach with applicability for architectureSIVS, inspired a . robotics Chapman 1990 developed perceptual , ( ) . mlman ' s visual routine theory Similar methods using visual routineswere by ' and also deployedin Chapman Agre s earlier system Pengi(Agre and Chapman , . the SIVS architectureControlinputsselectfrom 1987 . Figure7.8 depicts ) performed guiding the overallprocessing amongthe primitive visual operators on the substratesreferred to as early (retinotopic maps in the figure. The ) , ' in routinesare selected a task specific top- down mannerbasedon the agents , that . actionrequirementsThe operators havebeendefinedin SIVS extendthose by proposed Ullman: . Visual attentionand search whereto look within the scene : . Tracking following moving objectsthroughthe useof visual markers :

259

Perceptual Basis for Behavior - Based Control

(C )
) Figure 7.6 (continued

260

Chapter7

(A ) Figure 7.7 Fastregion SegmentationThis sequence . showsthe resultsof a fast region segmenter whenusedfor pathfollowing. It is particularlyusefulwhenroadedges weakandthe are fast line finder is not suitable Only the centralportion of the path is extracted obtain . to a more consistentsegmentation(A ) showsthe robot locatedon a gravel path whose . . edgesare coveredby grass (B) showsthe extractedregionand (C) the computedpath fit centerline . edges(least squares ) and the computed . Spatial properties : the ability to compute distances, angles and directions to objects within the world , thus constructing concrete spatial relationships . Activation : determining whether a selected region is bounded through spreading activation

. Others including varioushousekeeping markermanipulationoperators : and . This systemwastestedonly in a video gameenvironment thusbypassed and muchof the difficulty of dealingwith real perceptual . algorithms Nonetheless the overallcontrol architecture faithful to Ullman' s approach providesa is and solid steptowardsoperationalizing on a real robot. it Reeceand Shafer( 1991 implementedvisual routinesfor potential use in ) robotic driving at CarnegieMellon University Their system called Ulysses . , , ' s work 1987 and usedfourteen was inspiredby Agre and Chapman routines ( )

261

Basisfor Behavi9rBasedControl Perceptual

(B)

(C) 7.7 ) Figure (continued

262

>a C : 0~

EARL MAP EAR MA EA MA

VISUAL OPERATOR
.

CONm OL SYSTEM

Figure7.8 SIVSperceptual architecture . control information for selectingsteering (table 7.1) to provide the necessary and speedmaneuvers tactical driving. It was testedon a traffic simulator for andshowed that specialpurpose routinesreducethe tacticaldriving perceptual task' s computational ordersof magnitude . complexityby several
7.4. 3 Perceptual Classes

The notionof perceptual classes particularlyusefulfor describingperceptual is in it , requirements behaviorbased systemsbecause permitsdefiningperceptual ' tasksbasedon the agents needs The partitioning of perceptualeventswithin . the world into equivalence classes basedon the needsof a motor action: is

263

Perceptual Basis for Behavior -Based Control

7.1 Table
Routinesin illysses Perceptual Find-current lane -next-car-in-lane Find Find-next-sign Find-path in-intersection Find-intersectionroads Mark-adjacentlane Find-next-lane marking Find-back facing-signs Find-next-car-in-intersection Find-crossingcars Track-lane Profile-road

-signs Findoverhead Findsignal

obstacleor nonobstacle road or nonroad landmarkor anything else moving ; ; ; . object or stationary The perceptualtask directs the appropriatesensory mechanism the consumingmotor behavior By channelingthese to . processing tasksdirectly, the ability to execute themin parallelon separate perceptual pro cessors obtained thusenhancing is . , computational performance Donald and Jennings( 1991a 1991b have contributedformalizationsfor , ) this concept They view the designof perceptionasthe constructionof recognizable . sets placesor things the robot is capableof perceiving Taskdirected : . for perceptionare a naturalconsequence because robot can be the , strategies whatit shouldencounteri.e., a definition ( providedwith expectations regarding of the characteristics the perceptual of classbeing sought (Donald Jennings , , ) and Brown 1992 . Sensingis considered mappingof the robot' s view of the a ) world onto the setof possibleinterpretationsdefinedby the perceptual classes , themselvesCareful constructionof the perceptual . classes makethis map can . ping easierthan in an unconstrained interpretation In defining the perceptual classeswhat the robot is pernrittedto senseand understand denoted Correlating is . , the resulting perceptual classes the needsof the behaviorspernrits to the extractionof information to be limited to only that requiredfor a particular task. The notion of perceptualequivalencewherebya large disparateset , , of uncertainsensoryreadingsis reducedto members particular perceptual of classes renderscomputationaltractability to the otherwiseunduly complex , . interpretationproblem The notion of information invariantsas the basisfor has this work evenfurther, pointing towardsthe eventual equivalences extended of development a calculuswherebyrobot sensorsystemscould be evaluated es ) analytically (Donald 1993 . This work involves the use of approach the reductionof one sensorinto employing computationaltheory pernritting . another The perceptual classmethodpernritsthe minimizationor elimination equivalence of mapconstructionin a mannerstronglysupportive purely reactive of

264

Chapter?

has ( ) systems Donald andJennings1991a. This research focusedprimarily on landmarkrecognition(wherethe robot is) ratherthan on which objects spatial afford what actions Nonethelessthe principles appearreadily extensibleto . , this broaderproblem .

7.4.4 LightweightVISion
Horswill ( 1993a has forwardeda consider ) , ably less theoretical more pragmatic to . , but no lessimportantapproach the designof perceptualalgorithms The overall approachdubbedlightweightvision, focuses specializingindividual on , es . perceptualprocess tailored to the behavioraltasksat hand Horswill that althoughthis specializationmight lend the appearance acollection of argues of disjointed ad hoc solutions there are principled meansby which , , thesespecialized modulescanbe analyzed produceboth generalization to and . This is accomplished part by making explicit the assumptions in potential reusability that underlie the applicationof a perceptualalgorithm for a given taskenvironment . . Lightweight vision incorporatesboth task and environmentalconstraints Theseexplicit constraints the basisfor designof the specialized provide perceptual module The claim is madethat for mostreal world task environment . pairs, a potentially large numberof perceptualsolutionsexist. Each solution within that solutionspaceis referredto asa " lightweight" system . vision is loosely related to Donald' s methodsin perceptual Lightweight classes(section 7.4.3). An equivalence class is concernedwith equivalence the theoreticalequivalence multiple perceptualsystems a task' s context of in . Horswill acknowledges manysystems that providean equivalentsolutionfor a class(asdefinedby a taskenvironment it becomes engithe ); givenperceptual ' neers goal, however to definea low-cost, highly efficient perceptual solution , within that space potentialsystemsThus lightweight vision providesa design of . for methodology constructing specialized perceptual algorithmsfor usein behaviorbasedrobotic systems(i.e., thosethat do not require reconstruction of the environment someabstract in form representational ). is accomplished the explicit declarationof constraintsimposed Design through on the perceptualtask. Habitat constraintsrefer to the set of environments within which the specialized . lead systemwill operateTheseconstraints to the formulation of a computationalproblem that then lends itself to optimization . Polly (figure 7.9) is a robot whosepurposeis to roam through the corridorsof the MIT AI lab andprovidetoursfor visitors asneededA partial . list of the percepts in requiredfor this taskappears table 7.2.

265

-Based Basis Control Perceptual for Behavior Table7.2 Partial of Percepts withinPolly list used . openleft? ? openregion -floor ? light -ahead ? person
open right? blind? dark-floor? ? persondirection blocked ? vanishingpoint? farthestdirection? -ahead wall ?

Table7.3 elaborates habitatconstraints the environment the that providesfor of depthrecovery(needed obstacleavoidance and vanishing for ) computation for ) point (needed headinginformationfor navigation. Theseconstraints pose tractablecomputational to problemsthat in turn lend themselves efficient, low. cost lightweight solutionsfor eachof the perceptual modulesrequiredfor the tour. laboratory ' Polly s architectureconsistsof a coupled low- and high-level navigation . system The low-level architectureis composedof speedcontrol, corridor -style follower, wall follower, andballistic turn behaviors using a subsumption arbitrationmechanismThe high-level navigationsystemprovidesdirections . as to whereto go next in the tour and conductsplacerecognitionto assistin verifying wherethe robot is, at variousplaceswithin the tour script. , Polly is a very robust system having given in excessof 50 tours in two different laboratories( MIT and Brown University). Perceptual specialization alsoknown aslightweight vision) hasenableda small, computationallyweak ( robot to conducta complexvisual navigationaltask by permitting the design of algorithms that fit a specific task environmentpair. The vision modules . may alsobe reusedin similar tasksandenvironmentsThis systemhasclearly demonstrated pragmatismof modular vision in behaviorbasedrobotic the . design

7.5 ACTIONAND PERCEPTION

modulesand motor behaviorscan be bundledtogetherin different Perceptual . To someextent one may questionwhetherperceptionis driving action , ways haveresulted : or vice versa 1 different forms of behavioralperception . \\10 . action-orientedperception in which behavioralneeds determine perceptual the , used and , strategies . activeperception in which perceptualrequirements dictate the robot' sac, tions.

N :

Chapter?

7.9 Figure
. Polly, a robotic tour guide (Photograph courtesyof RodneyBrooks.)

267

Basisfor BehaviorBasedControl Perceptual Table 7.3 1\ vo setsof exemplarhabitatconstraints their usein Polly' s task. and Perceptual Need Depthrecovery for navigation
Constraint

Computational Problem Optimization useheight in image (indoor level floor) usetexturealgorithm line finding usepixels as lines low-costdetector ID clustering

GrouRd plane assumption texture Background constant(carpet )

Corridor vanishing long corridor edges point for heading (office building)

Strongcorridor edges edgedetection known cameratilt clustering

" To someextent this is a " chicken and egg question relating to the origin , , of perceptionin animals Did perception : result from the requirements locomotion of ? , or did the evolutionof perceptionenablelocomotorycapabilities We leavethis questionfor othersto answerand look insteadat the impact of both of theseapproach on the designof behaviorbased es robotic systems .

7..5 Action-OrientedPerception .1
Action-oriented perceptionrequires that perceptionbe conductedin a topdown manneron an as neededbasis with perceptualcontrol and resources , determined behavioralneeds As statedearlier this is in contrastto more . , by traditionalcomputervision researchwhich to a largeextenttakesthe view that , perceptionis an end in itself or that its solepurposeis to constructa modelof of the world without any understanding the needfor sucha model. Thesenonaction -orientedstrategies burdena robotic systemwith unnecessary processing . that requirements canresult in sluggishperformance . Action-orientedperceptionis not a new concept As we have seen it has , roots in both cybernetics(Arbib 1981 and cognitive psychology (Neisser ) 1976 . The underlyingprinciple is that perceptionis predicatedon the needs ) for of action: Only the informationgermane a particulartaskneedbe extracted . from the environment The world is viewed in different ways basedupon the ' . agents intentions : Action-orientedperception manyaliases selective has ( perception Simmons 1992 purposive vision (Aloimonos and Rosenfeld 1991 situated vision ), ), ) ), (Horswill and Brooks 1988 and task orientedperception(Rimey 1992 are

268

Chapter7

a few. The underlying thesisfor this methodis that the natureof perception is highly dependent the task that is being undertakenThe task determines on . the perceptualstrategyand processing . in , required This approach developed , conjunctionwith reactiverobotic systems tailors perceptionto fit the needs of individual motor behaviors Insteadof trying to solvethe so-called general . vision problemby attemptingto interpretalmosteverythingan imagecontains , an advantage gainedby recognizingthat perceptualneedsdependon what is an agent is required to do within the world. Avoiding the use of mediating in global representations the pathwaybetweenperceptionand action removes a time-constraining bottleneckfrom robotic systems . In action orientedperceptionthe motorbehaviors , providethe specifications for a perceptual : from environmental process what mustbe discerned sensing and constraints to whereit may be located Focusof-attentionmechanisms as . playa role in directing the perceptualprocessas to where to look, and expectations of ) provide clues(e.g., models as to what the appearance the event the in beingsoughtis. How to perceive desiredeventis captured the perceptual ' modules computational . process Behaviorbasedsystems can organizeperceptualinformation in threegeneral ), ways: sensorfission ( perceptual channeling action oriented sensorfusion and perceptualsequencingsensorfashion (figure 7.10). Sensorfission , ( ) is straightforward a motor behaviorrequiresa specificstimulusto producea : moduleis createdthat channels output its , responseso a dedicated perceptual to the behavior A simplesensorimotor . circuit, numerous of directly examples which we havealreadyencountered chapter3, results in . Action-orientedsensorfusion permits the constructionof transitory representations . (percepts local to individual behaviorsRestrictingthe final percept ) ' to a particularbehaviors requirements contextretainsthe benefitsof reactive and control while permitting more than one sensorto provide input, resulting in increased robustnessSection7.5.7 explorestheseissues . further. Sometimesfixed-action patternsrequire varying stimuli to support them over time and space As a behavioralresponse . unfolds different sensors or , different viewsof the world maymodulateit. Perceptual allowsthe sequencing coordinationof multiple perceptual algorithmsovertime in supportof a single behavioralactivity. Perceptual algorithmsare phasedin and out basedon the ' s needs the environmental and contextin which it is situated The phrase . agent sensor this notion of the significanceof differing fashion coarselycaptures . perceptualmoduleschangingover time and space Section7.5.6 studiesthis of in aspect coordinated perception moredetail.

269

J PerceptuaBasisfor BehaviorBasedControl Percept 1 Percept 2 Percept 3

.'~ ,,~ " ~, .1 ~~"~~ I ~Response-1 ~ t ~ ~ .L~ -.....- -J ~Response-2 ~Response-3 , ,~ ~ A ()

Percept . I.._ ~ _ I .._ ~ Response - 1

Percept 1 - ~ Percept 2 =1~ ~ Percept3 / ' Percept 1 ~ Percept 2 . Percept 3

(B ) -of one (C ) -1 ~Response

Figure 7.10 : Dimensionsof action orientedperception (A ) Sensorfission- multiple independent motor behaviors each with its own perceptualmodule ( B) Sensorfusion multiple ; , a single perceptualmodule within the context of submodules supporting perceptual a single motor behavior and (C) Sensorfashion multiple perceptualmodulesare ; . can within the contextof a singlemotor behavior The different dimensions sequenced . be composed together

es. ) Crowley et ale( 1994 havetried to formalize reactivevisual process In creates onto an action space their approach a setof virtual sensors , mappings controllers The behavioritself . determinedby actuatorsand their associated . providesthe mappingfrom perceptualspaceonto action space Supervisory of control pennits the selectionand control of the sequencing individual perceptual es (chapter6). process in the style of hybrid architectures
7.5.2 Active Perception Active perception focuses primarily on the needs of perception , rather than the needs of action. The question changes from the action -oriented perspective of " " How can perception provide information necessary for motor behavior? to

270

7 Chapter " How canmotor behaviors " supportperceptual activity? Thesetwo viewpoints are not mutually exclusive indeed activeperceptionand action orientedperception ; , are intimately related What the agentneedsto know to accomplish . its tasksstill dictatesperceptual , requirementsbut activeperceptionprovides the perceptual es process with the ability to control the motor systemto make its task easieras well. In contrast Blake ( 1995 characterizes nonactiveperceptual , ) that rely upon a single vantagepoint as the equivalentof a strategies " ." seeingcouchpotato ' , , Accordingto PahlavanUhlin , andEklundh s usefuldefinition ( 1993 p. 22), "An activevisual its systemis a systemwhich is ableto manipulate visual parameters in a controlled mannerin orderto extractuseful dataaboutthe scene in time andspace Bajcsy' s ( 19~8) seminalpaperon activeperception ." charac terizesit as the applicationof intelligent control to perceptionusing feedback from both low-level sensoryprocess and high-level complexfeatures Useful es . a priori knowledgeof the world may be either numeric or symbolic and encodessensormodels and expectationsActive perceptionsharesthe view . -orientedperception with action that sensory modulesarea principal commodity in . is , with perceptual goalsembedded thesemodules Active perception thus definedas an intelligent dataacquisitionprocess intelligent in its useof sensors , . guidedby feedbackanda priori knowledge Work at the University of Toronto epitomizesthis approach Wilkes and . Tsotsos( 1994 definebehaviorsfor controlling a video camerato extractthe ) information for objectrecognition Threespecificbehaviorsare defined . necessary for this purpose imageline-centering providing rotation to the camera : , to orient the objectvertically within the image(rotationalcontrol) ; imagelinefollowing, providing translationsparallel to the image plane to track object features tangentialcontrol); andcameradistancecorrection providing translation , ( the optical axis, in essencezooming (radial control). Because , along thesebehaviorspermit the camerato control its position in space it can use , active explorationstrategies confirm or annul hypotheses to from generated the candidateobject modelsavailableto the system Figure 7.11 captures . the . its goal, moving aboutat the systemarchitectureThe camera activelypursues endof a robotic arm to continuallyobtainmoreadvantageous viewpointsuntil an identificationis achieved The systemwas testedsuccess . fully in complex scenes sufferedfrom high levelsof occlusion The ability to havemultiple that . views guidedby feedbackfrom the recognitionprocess the reduces computa tional complexitysubstantially overwhat may otherwisebe an impossibletask whendatais from only a singleor an arbitrarycollection of prespecified CaIn

271

Basisfor BehaviorBasedControl Perceptual

Rotational Motion LowLeve Corresponden 1 8_8_ 1--- " Image --..... Based ~~ ~ r Features Tracking Radial Motion Motion Tangential Index of Object Candidates New View if uncertain ( ) . Object Identity If certain ( ) ---... Action Controller

7.11 Figure
. , Exemplaractivevision architecture

era viewpoints Similar research also beenconducted the University of . has at Wisconsin(Kutulakos Lumelsky andDyer 1992 . , , ) Ballard ( 1989 hassummarized importantcharacteristics activevision the of ) which he refersto asanimatevision): ( . An activevision systemhascontrol over its camerasit can search movein, : , focus, andso forth. change . Active vision canmovethe camera preprogrammed in , stereotypical ways if needed . . Image featurescan be isolated and extractedreliably without the use of models . . Alternate coordinatesystemsbecomeavailablefor exploration insteadof , merely an egocentricframe of reference as is often the case with purely reactive systems Action and perceptioncan be conductedwith an object . centered frameof reference with respect someindependent or to featurewithin the world. . Fixation points provide the ability to servowithin the visual frame of reference . . Object centeredcoordinatesystemshave inherentadvantages to their due invariancewith respectto observer motion. Active vision has been treatedin severalother books (Aloimonos 1993 ; ChristensenBowyer and Bunke 1993 to which the interestedreaderis referred , , ) . Our discussionnow returnsto a more behaviorbasedaction oriented ' s intentions in perspective which an agent directly determineperceptualrequirements .

272

? Chapter

7.5.3 What About Models ?

A priori knowledgeof the world can playa significantrole during perceptual . processingParticularcasesfavor the appropriateuse of this form of knowledge . Theseuses includeprovidingexpectations perceptual for es: process what -of-attentionmechanismswhereto to look for (section7.5.4); providing focus : look for it (section7.5.5); sequencing perceptualalgorithmscorrectly: when to look for it (section7.5.6); and initially configuring sensorfusion mechanisms : how to look for it (section7.5.7). Theseuses areentirely properwithin , the context of motor behavior It is not so much in what form the representation . but es appears rather how it is usedthat distinguish the action oriented from other methods . approach Thereis alsothe issueof perceptsevents within the environment , , perceived and how they should be encoded if at all. The dictionary definesa percept , " as " an impressionof an object obtainedby use of the senses sensedatum ; to 1984 . A questionarisesas to whetherit is at all necessary create ) ( Webster an abstract suchan eventor impression The pure . encoding representation reactivistwould arguethat the signal on a sensorcircuit wire is the encoding of the sensoroutput and neednot be mademore persistentor abstractthan that. Certainly stimulusresponse in , pairscanbe characterized this manner so from a perceptualperspective A recent ? add any additional representation why the to ), response this position (Agre andChapman1990 advocates useof 's which capturethe agent relationshipwith its transientdeictic representations environmentat any particular point. As an examplefrom their Pengi system (Agre and Chapman1987 a video gameenvironmentincluding variousanimal ), . , agentssuchas penguinsand bees would include the-bee I -am-chasing ' s intentionsand the situationin which the The agent agentfinds itself define . the particularbeein question Pengiusesvisual routines(section7.4.2) to extract the relevantdatafor the situatedtask pair. Thesefleeting action oriented in are as representations constructed necessary supportof particularbehavioral however were not physically tied to needs The routinesusedin this system . , , . an operatingsensor system ' embodied theseideas , ) Hayhoe Ballard, andPelz s ( 1994 work at Rochester in a real system Camerafixations provided the contextualbindings for the . . Perceptual information insteadof being stored as a deictic representations , . createdare relevant , history is requiredimmediatelyprior to its use Descriptions , representations only to a particulartask avoidingthe memoryconsuming . neededshould traditional geometricmodels be stored To someextent the ,

273

-Based J for Control PerceptuaBasis Behavior

Memory -

Sensors

- - - - - --- 1

Perception

Action

7.12 Figure Control . system deicticpatialmemory using s

environmentitself servesas the representational storehouserequiring only , suitableperceptualalgorithmsto exttact the infonnation as neededThe system . was testedin a simple blocks world environment with simple perceptual , routinesto GetColor andGetLocation movethe blocksfrom onelocationto to . another Cognitivemodelsof humanperfonnance . motivatedthis study which , " concludedthat the visual representations humansuse are " extremelyscant " minimal information carried over between " It can be with fixations . argued that theseconclusions serveassolid guidelinesin the useof visual representations for behaviorbasedsystems well. as ' s 1994 research the Brill ( at ) University of VIrginia arguesfor the extension of deictic representations includelocal spatialmemory The impetusis to . to provide information that goesbeyondthe limits of reflex action, providing to the control systemsomehistory of eventswithin the world that go beyond ' the sensorysystems immediaterange Deictic representations . again serveas the basisfor the basicconstructsbut addedmemoryprovidesthe robot with a , broaderperceptual layout. The systemthus can both act from immediatesensory data as in a typical reactivesystem but can also reactto spatialmemory , , constructed a task orientedmanner figure7.12 reminiscent the structure in of ( ), seenin the modified action perceptioncycle discussed earlier (section7.5.1). ' It alsoresembles for . Paytons methods internalizingplans(section5.2.2 2) but -deriveddeictic modelsasopposed incorporatinga is concerned with sensor to ' . frameworkhasbeenexploredonly in priori mapknowledge Brill s conceptual a preliminarymannerthusfar. Wenow moveon to investigating how informationaboutthe world, however it may be represented derived canhelp constrainthe inherentcomplexityof or , perceptualinterpretationas we examinethe intertwinedroles of expectations andfocus of-attentionmechanisms .

274

Chapter

7.5 Expectations .4
resources perfonn effectivelyin to enablelimited computational Expectations real time. Psychological e.gNeisser 1976 andneuroscientific e.g., Spinelli ) ( ( ' reaffinnedexpectationsrole in biological systems 1987 studieshaveconsistently ) es . Expectations tell perceptual can process at leasttwo things: whereto event and how that particularperceptual event look for a particularperceptual can will appear Temporalcontinuity and consistency . assumptions enablea to restrict the possiblelocation of an object from one point in time to system in the next. Things generallydo not disappear oneplaceandmaterializein another ' motion, allows . This fact, coupledwith knowledgeof the systems ego . to processing significantconstraints be appliedto sensory can One way expectations be exploitedis by a recognitionthat much perceptual : activity involvestwo majorcomponentsrecognitionandtracking. The basedon the or discoveryphaseis often a model driven approach recognition anticipationof what the object in questionmight look like. Its representation can or characteristics be retrievedfrom memory or may be hard wired into . a specific object recognition behavior In somecases as in the caseof obstacles , a recollection of how specific obstaclesactually appearneednot be , that invokedbut rather a recall of the affordances makean obstaclean obstacle in is . Oncerecognitionis achievedthis phase abandoned favor of the newly , of arrivedperceptsforming the basisfor tracking the whereabouts the object . in question This tracking phaseis computationallyfar less demandingthan are derivedfrom perception moreimmediate the , discoverybecause expectations -tenDrecall or a more thanthoselong andhencemorereliableandaccurate . modelprovides abstract ' that an agents instruction is , Dlustratingthis processby example suppose to turn right just past the first oak tree. If the agenthas previously traveled the route, somementalimagemay exist of what the particulartree in question . looks like and whereit might be located This recollectionmay be imperfect , conditions(night insteadof beenfonned underdifferentenvironmental having , , day fog versussunshine etc.). Additionally, the tree may havegrown, been . , damagedor beenalteredin somemannersincelast observedThus we must assume our modelcan provideat bestonly limited knowledgeof how the that . treeshouldappear It is also entirely possiblethat the agentmay neverhavetraveledthe route . before and thus somegenericmodel for an oak tree will needto be invoked , for demand recognition asfewerconstraints This forcesgreatercomputational , . can be placed upon the interpretationof incoming sensordata Nonetheless

275

J PerceptuaBasisfor BehaviorBasedControl

somerestrictionscanbe placeduponthe sceneinterpretation regardingwhere oak treesare likely to occur in generaland their typical characteristicse.g., ( , , , color, breadth leaf shapeetc.). height that has the Let us now assume . the recognitionphase success discerned fully oak tree in question The task remainingis simply to track the position of this . . tree until the next changein motor activity, which is the right turn, can occur We no longer needto continually distinguish what type of tree it is, rather all that is neededis to track its position so that the turn can be madewhen now concentrates on the opportunitypresents itself. The perceptual processing featuresand not on somevagueor the tree' s immediatelyextracted perceptual for . Color or shape outdated may be adequate trackingpurposes representation . that in a settingwherethe tree is solitary By exploiting the assumptions the is anchored placeand its reflectance relatively constant it can in tree remains , . readily be trackedas the robot movesabout Although lighting variationsand ' of can other environmental , changes and do occur the expectations the tree s of trackedfeaturesare continuouslyupdatedlargely independent any a priori of sensorbasedrepresentation the tree is maintained model. No independent at or needed the reactivelevel. This form of adaptive trackingremainsin effect or until eitherthe taskis completed the recognitionoccursthat someperceptual . criteria hasbeenviolated forcing a recall of the recognitionphase , Le at Researchers the Universityof Maryland( Waxman Moigne, andSrini, vasan1985 Waxmanet al. 1987 havelookedat the recognitionandfollowing ; ) land vehicle project. Their of roadwaysas part of the DARPA autonomous : for roadfollowing is brokendown into two distinct phasesbootstrap process 's . image processingand feed forward processing The bootstrapsystem purpose ' s locationwithout informationregardingthe vehiis to find the road prior cle' s position. Oncethe roadhasbeensuccess identifiedandthe vehicleis fully in motion, a feed forward strategyis employedthat relies heavily on the inertial . guidance systemfor deadreckoning Wherethe road shouldappearwithin and is predictedbasedon its last appearance the motions sensed the image by the INS. Thesepredictionsrestrict the possiblelocation of featuresso that on centered their expected can positions processing be limited to small subwindows . , reducingthe computationdramatically The actual constraintsbased of on error studieshavebeensetat 0.25 metersand 1 degree orientation fine -cost systemsTo maintain . for inertial guidancebut impracticalfor most lower is thesetolerancesan expensive -tilt mechanism requiredaswell. Line extraction , pan , combining evidencefrom multiple image subwindowsto yield the , long parallellines of the roadsidesis the principal featureusedfor roadidentification .

276
7.5.5

Chapter? Focus of Attention Focus of attention , closely related to expectations, provides infonnation regarding where an age~t should expend its perceptual resources, guiding where to look within the image or determining where to point the sensors This in . essence is a search reduction strategy. The problem can be phrased as how to prune the search space within which the desired perceptual event appears. Several techniques use hardware strategies such as foveated cameras or focus control ; others exploit knowledge of the world to constrain where to look . We first examine the strong evidence supporting attentional mechanisms in human visual processing. The advantages from a computational perspective are presented , followed by examples of both hardware and software systems these issues have influenced.

7.5.5.1 TheRole Attention of

in Human Visual Processing

Vision in naturalenvironments confrontsthe observerwith a largenumberof stimuli within the field of view. Biederman( 1990 claims that there potential ) are at least three reasonsattentionalmechanisms must selectone entity for observation from the manypotentiallyavailable : . The eyeitself is foveated the retinacontainsonly a smallregion (the fovea ; ) in capableof resolvingfine detail. This is manifested the spotlight metaphol ; in which information is filtered so that attentionis paid only at the centerof attraction(Olshausen Koch 1995 . and ) . Shifting attentionfrom one region to anotherdoesnot necessarily require and . eyemovements thuscanbe doneat very high speeds . Serial shifting of attentionprovidesthe ability to integrateand usedifferent features suchas color and shapeThis serializationhascomputational . advantages , as a consequence the highly consttainedmannerin which a sceneis of . explored Culhaneand Tsotsos( 1992 at the University of Torontodeveloped strategy a ) to reducethecomputational of visual search machinevision. in complexity An attentional" beam" a localizedregion within an image is usedwithin a , , " multilevel abstractionhierarchythat selectivelyinhibits irrelevantimage regions variouslevelsof visual processing (figure 7.13 . The hierarchyrepresents ) , with the attentionalbeamappliedto the most abstractlevel. This in turn controlslow-level processingthusreducingdramaticallythe amountof image , . processing required

277

Basisfor BehaviorBasedControl Perceptual

Spotlight Attentional

Most Abstract Layer

Least Abstract Layer

j Zones Effective Inhibition I t Actual Pass Zone

7.13 Figure Attentional (after beam CulhaneandTsotsos1992 . )

CulhaneandTsotsos characterize their methodasa " continuous reactive and " This mechanism . systemfocusesprimarily on early vision and seemsmost for appropriate taskssuchastargettracking. After the initial targetlocationis determined attentionconstrainsprocessing lower regionsin the hierarchy at , by inhibiting computationin regions outside of where the target would be to within the nextincomingimage This modelcanalsohandle . expected appear via scanpath methods(Starkand Ellis 1981 (section5.3) . ) object recognition A prioritized searchfor different featuresat different spatiallocationswithin an imagecan be conducted easily by serially pointing the attentionalbeamat different imageregions .

7.5.5.2 Hardware Methods for Focus of Attention

. Hardwaresolutionsfor providing attentionhavealso beenstudied One strategy the is to embed spotlightnotiondirectly into the hardwareusinga foveated , . camera similar in principle to the eye from the onset Kuniyoshi et al. ( 1995 , , ) reviewsseveral suchmethods where , . multiple imagesareacquiredusinga zoomlens sequentially , focusingin on the objectbeingattended to. . multiple cameras differing fields of view (e.g., wide-angleandtelephoto use . lens while coordinatingthe cameras )

278

? Chapter
. a CCD chip is designed an artificial retina with a dense as foveatedregion. . a speciallydesigned lens is usedwith a conventional CCD arrayto produce a foveatedspacevarying image . has hardwareconsistent Although progress beenmadein developingspecialized with the eyeandpotentiallycapable exploiting comparable of focus ofattentionmechanismslittle work to datehasincorporated them into complete , robotic systems . 7.5.5.3 Knowledge -BasedFocus or-Attention Methods Knowledgeaboutthe world can also guide the applicationof attentionalprocessing . In research conductedat the University of Chicago Fu, Hammond , , and Swain ( 1994 havedeveloped methodsto link regularitiespresentin the ) World example in everydayworld with perceptualactivity. In their Grocery , which an agentis shoppingfor food at a supermarketdomain specificknowledge , is exploitedto guide perceptual . , processingFor example foods are typically cereals meats within types brands , , , groupedtogetherby types(soups ); tendto be colocated big itemsare storedlow on the shelvesperishable items ; , in freezers andso forth. This heuristicknowledgegenerates as , expectations to wherein a storeto find certainitems. Perceptual activities are then structured in a mannerthat yields the desiredoutcome finding the food item on the shelf. : Visualroutinesin this system includea type-recognizer an item-recognizer and built from three simple and fast vision algorithms Theseroutinesare highly . -environment which the shopper in specificto the task ( operatesan imagedatabase of a grocerystore . By exploiting the regularitieswithin this domain very ) , simpleperceptual algorithmscan accomplishwhat would otherwisebe a very difficult task. Anotherapproach to introducedeep causalknowledgeaboutthe world to is , focus attention In a blocks world exampleusing the BUSTER systemdeveloped . at Northwestern , , , ) University Birnbaum Brand andCooper( 1993 applied causalanalysisto explain why a complex configurationof blocks is capable of remainingstanding Expectations . of regardingthe causes physicalsupport drive the investigationallowing attentionto be appliedto regionsof the image , most likely to provide evidencefor an explanation Attention is first directed . to lower partsof the image wherephysicalsupportbegins After establishing . , , , supportat theselower regions imageprocessing proceeds upward providing as explanations to why or how a particularblocks world configurationcan indeed remainstanding This yields a visual attentionaltracethroughthe image .

279

Perceptual Basis for Behayior -Based Control

not unlike the scanpaths discussed section5.3. Here, explanationdrivesvisual in ' attention The agents goal is to determinewhy something the way it is. : is Causalsemantics will be embedded a completerobotic system in eventually , andthe resultingprogress will certainlybe of interest .
7. 5.6 Perceptual Sequencing

It is entirely possible and in many instances , , highly desirable to havemore thanoneperceptual with algorithmassociated a singlemotorbehaviorat different . with multiple parallel stages during its activation Sensorfission is concerned concurrentalgorithms and sensorfusion is concerned with , independent relatedconcurrentperceptualalgorithms Sensorfashion . combining multiple with the sequencing (perceptualsequencing on the other hand is concerned ), , of multiple relatedperceptual . algorithms Sensorfashion an aspectof action orientedperception recognizes the that , , for evena single motor behavioroften changeover perceptualrequirements time and space For example different perceptionis required to recognize . , an object when it is far away than when it is close Often, entirely different . cuesmay be usedover the courseof a singlebehavior . perceptual Assume for example thata mobilerobot' s taskis to dockwith a workstation , , in a factory (Arkin andMacKenzie1994 . One of the behaviorsnecessary for ) this task is Docking which providesthe response , regardinghow to moverelative to theperceived workstationstimulus Because operationis carriedout . this over a wide rangeof distancesfrom asmuchas 100feet awayto immediately ( in front of the workstation no singleperceptual for ), algorithmis adequate the task. Instead four distinct perceptual . , algorithmsarecoordinated sequentially When the robot is far awayfrom the workstation, the limitations of the video lens make it impossibleto discernthe dock' s structure The first perceptual . cueseither from lighting conditionsat the workstation or from motion algorithm a , to generate hypothesisabout the workstations location. The docking behaviorinfluences robot andit startsmoving towardthe hypothesized the target . As the robot approach the workstation, it must positively identify it. A es more computationallyexpensive , algorithm exploiting a spatially constrained versionof the Houghtransform(a model basedobjectrecognitionalgorithm ), is usedto makea positive identification confirming that the perceptualevent , in questionis truly the workstation. As soon as the workstation is positively identified an adaptive is , , trackingmethodology used basedon regionsegmentation . At the final stages docking ultrasound of , positionsthe robot.

280

Chapter7

The primary issuesin perceptual are sequencing when to useeachof these andhow to determinethe besttime to switch perceptual . algorithms strategies In our example while the robot is far from the dock, we usethe long-rangedetection , , algorithm headingdirectly towardsthe dock; whenwithin an expected is , recognitionrange the model basedstrategy usedwith the resultingcuetriggering the approach the docking behavior As the rangecloses a transition of . , ' from vision to ultrasound occursasthe cameras field of view becomes useless at closerange . Relatedto this is the notion of a perceptualtriggel; a perceptualprocess that invokesa changein either the behavioralor perceptual stateof the robot. when a perceptual The behaviorbasedrobot' s control systemis reconfigured , trigger fires, changing either the set of active behaviors the set of active es , perceptualprocess , or both. These triggers can be quite simple such as proximity information or detectionof a color or motion (e.g., a cloak waved in front of a bull), or they can be more complex like the dock recognition , mentionedabove We have seen in section4.4.4, how perception . , algorithm for canbe usedto trigger different behaviors a foragingtask. An experimental for our exampleshowsthe robot using four different run schemas the courseof traversinga distancefrom thirty feet to in perceptual one half foot. A finite stateacceptor(FSA) (figure 7.14) is usedto express in the relationshipsamongthe individual perceptualstrategies the contextof . the docking behavior Allowing failure transitionsto be presentwithin the robustness . Figures 7.15 and7.16illustratethe exampledocking FSA ensures run. At the University of PennsylvaniaKosecka Bajcsy andMintz ( 1993 have , , , ) developeda similar approachbasedon discreteevent systems(DES). This and control to enableand disablevariousperceptual methodusessupervisory . motor events Abrupt sensoryeventstrigger different strategiesBogoni and : ' to ) Bajcsy s ( 1994b extensionof the DES approach the investigationof functionality usesa supervisoryoverseerfor controlling, arbitrating and fusing , for evidencefrom multiple sensoryvantagepoints. In their work, developed usesFSAs to control multiple sensors the manipulationdomain an overseer , 's sensorsin to ( (figure7.17). The overseer stateis reflected the two subordinate the exampleshown tactile and vision). Tactile sensingprovidesonly contact , . vision canbe informativefor all states informationwhereas morecomplexmappingfor a piercingtask 7.18showsa consider ably Figure has (detectingwhena screwdriver pierceda styrofoambox).

281

Basisfor Beh~vior-BasedControl Perceptual

-n-TContro - I --led - -, ', -: 'I-r--------Ia "Ballistic rt ,- ': nn: I , ''I ,

n : normal t : terminal r : recoverable error f : fatal error a : all Obstacle

q 0 : ballisticperception q 1 : exteroceptive cue q 2 : adaptivetracking

q 3 : finalpositioning termination q 4 : normal

q 5 : abnormal termination

Figure 7.14 FSA encodingtemporalsequencing usedfor dockingbehavior .

~" .._ ~ \---_ ~ . -_ .

Robot

Light ~~t 3-..-~-: -6 . ..-l _,

Dock

Figure 7.15 Traceof dockingrun with an obstacle . includedocking noise , presentActive behaviors , and avoid static obstacles .

N ~

Chapter7

'= -

" " , . , , ; . . e

~-

7.16 Figure

283

Control Basisfor Beha~ or--Based Perceptual

(F ) ) Figure 7.16 (continued for ; Sequencing Docking: (A ) Phototropiclong-rangedock detection (B) Perceptual model based dock recognition (C) Adaptivetrackingusing regionsegmentation Successful ; ; ; ( 0) Loss of region due to obscuration (E) Rerecognitionusing model based and (F) Final adaptivetracking image followed by final positioning using ; technique . ultrasound 7.5.7 Sensor Fusion for Behavior - Based Systems
' A man with a watch knows what time it is ; a man with two watches isn t so sure. - Anonymous

' Sensor fusion s traditional role has been to take multiple sources of information , fuse them into a single global representation, and then reason over that representation for action , an approach at odds with the basic view of repre ' role in behavior -based robotics . This does not mean however that sentation s , , sensor fusion no longer has any place within behavior -based robots , for multiple sources of information can significantly enhance the way an agent acts within the world . Action - oriented sensor fusion advocates that sensor reports be fused only within the context of motor action and not into some abstract, all -purpose global representation. Fusion is based on behavioral need and is localized within the perceptual processes that support a particular behavior.

284

Chapter7

OVERSEER

VISION

F
A: Approach N: NearContact F: Failed Contact C: Contact

F
FORCE

Figure 7.17 's In this simpleexampleof the DES approachthe overseer states mappedinto two are , sensorsvision and force, eachcontributingwhen it can. The shaded , areaindicatesno useful informationis availablefor that particularsetof states . Sensor fusion , in any case, is not as simple as it sounds; incoming evidence may be complementary (i .e., in support of other observations) or competitive (i .e., in contradiction ) . Further, the incoming evidence may be arriving at different times ( asynchronously), as some sensors take longer to process than others. Often there are qualitative distinctions in the nature of the information provided : Vision may yield color information regarding the presence of a soda can, a laser striper may yield shape data, and a tactile sensor the can' s surface texture. The information may also be coming from widely separated , viewpoints . Deciding what to believe is a complex task and the behaviorbased roboticist ' s goal is to provide a single coherent percept consistent with the incoming evidence. Pau ( 1991) considered the r <le of behavior in sensor fusion , recognizing that ? a behavior-based system would appear as a collection of subsystemsin support of a particular behavioral actor. His cognitively inspired approach strives to ' provide fusion support in the context of each individual agent s requirements, in contrast to the traditional method of constructing task- independent representations . Formalisms have been provided for specifying fusion processes relative to actors, but they have not been tested within the confines of a robotic system.

285

Basisfor Beha or-BasedControl ~ Perceptual

OV
POSITION
States : Operating S: Start G: Goal(Success ) AApproachD : Depart C: Contact (generic ) CI: Contact Insert ( ) CE: Contactextract ( ) E: Extract P: Piercing I: ToolInObject T: ForceSensed NF: No Force : NC NoContact Failure : States Fc: contact failure failure Fp: Piercing Fg: Goalfailure Fe: Extraction failure Fs: System failure FORCE

>z as 0

Figure 7.18 : The more complex piercing task requiresthe coordinationof three sensors vision, onto the individual from theoverseer andforce. The figure showsthe mappings position as states state the sensor . sensorsWhen the overseer , change well. changes

286

Chapter?

' Kluge and Thorpe s ( 1989 work at CarnegieMellon University took a ) more practicalapproach coordinatingmultiple sources information The to of . FERMI system usedfor roadfollowing, employsa collectionof roadtrackers , , eachtracking different featuresaboutthe road, with all providing their information to the robot controller for road following. Five different road trackers havebeendevelopedtwo edgebased : methods orientedto a particularfeature , anotherthat extractslinear featuressuchas road stripes a boundarydetector , . , usingcolor information and a featurematchingalgorithm All of the trackers ' provide information regardingthe road s location. Trackersare fusedusing a Hough basedmethodin which eachtrackervotesfor a particularlocationof a road spine(centerline. This voting process determines winning candidate the ) roadposition. Arbib, lberall, and Lyons ( 1985 were amongthe first to introducethe notion ) of a task dependent in . representation the contextof robotics Their work advocated coordinateduse of a set of perceptualschemas provide the the to information for a motor task. Murphy ( 1992 hasfurtheredthis notion necessary ) of coordinated schemas the basisfor action orientedsensor as perceptual fusion. Her model draws heavily on psychologicaltheoriesof sensorfusion to (Bower 1974 Lee 1978 and usesa statebasedmechanism control important ; ) sensorinteractionssuch as cooperation competition recalibration and , , , . subschemas dedicated individual sensors to suppressionContributingperceptual and funneled into a controlling parent perceptualschemaprovide the means expressing model. Murphy' s work coalesced the development for this in of the SFX architecture Murphy andArkin 1992 . ( ) SFX' s analogof the underlyingpsychologicalmodelhasthreestates : State 1. CompleteSensorFusion All sensors : cooperatewith each other in a valid percept . determining State 2. Fusion with the possibility of discordanceand resultant recalibration : sensors occurs of dependent perceptualsources Recalibrationof suspect ratherthanthe forced integrationof their potentiallyspuriousreadings into the derivedpercept . State 3. Fusionwith thepossibility of discordance a resultantsuppression and discordantperceptualsources Spuriousreadingsare entirely ignored by : of the . suppressing output streamof the sensors in question ) The task specificperceptual schemas usedfor fusion yield percepts directly related to the needsof a motor behavior Perceptualsubschemas . feed their . , parentschemawhich in turn supporthigher level schemasTies to the actual sensordataemanatingfrom eachsourceeventuallygroundthis recursivefor-

287

Basisfor BehaviorBasedControl Perceptual

REQUIREMENTS FROM MOTOR BEHAVIOR

EXCEPTION HANDLING

SENSOR DATA

PERCEPTFOR BEHAVIOR

Figure7.19 TheSFXarchitecture . mutation A parentperceptualschema . combinesthe incoming subschema information statisticaluncertaintymanagement using (Murphy 1991 ) techniques to producea perceptand a measure its belief to be usedwithin the motor of schema itself. . Figure 7.19 depicts the overall control flow within the SFX architecture The perceptualschemaand subschema are configuredprior to execution arrays 's during the fusion process investigatoryphasebasedon the activemotor ' behaviors needs This investigatoryphaseis analogousto the preexecution . of behaviors found in manyhybrid architectureschapter6). The configuration ( of sensor fusion similarly matches executionaspects the of performatoryphase reactivecontrol andproceeds without hierarchicalsupervisionAn observation . directed acyclic graph (oDAG) incorporates sensingactivity (algorithms the and sensors to generate necessary the behavioralpercept(figure 7.20) . This ) control scheme( Murphy 1992 has been developedto provide error ) unique = ry r" '-c., ~ capabilitiesin light of potentialsensor failuresor uncertainreadings .

288

Chapter7

OBJECTERCEPT P

/
ULTRASONIC RANGE PROFILE ,
Nearest Surface

VISIBLE LIGHT CCDCAMERA .

THERMAL IMAGE INFRARED CAMERA

~

/ \ rHeightPosition fX/ \ f -2 -3 1 Region FeatureFeatureFeature Region -2 -1

Feature Constellation

Figure 7. 20 the This oDAG represents generationof a percept associated with an object using from threedifferent sensorsultrasoundvideo, andthermalimagery : evidence . , gathered Features their associated and belief valuesarepropagated the upwardto generate belief in the overall percept .

7.6 REPRESENTATIVE EXAMPLES OF BEHAVIOR -BASED PERCEPTION for is Perceptual processing mobile systems notoriouslydifficult. Working in that are in motion a partially known and uncertainenvironmentwith sensors to and subjected bouncing the perceptual , systemmust provideinformationto . a robot that is both useful and accurate In this sectionwe investigatethree tasks roadfollowing, visual tracking, androbotic headcontrol. : representative

7.6.1 RoadFollowing
. fonD of amnesiaEverythingis to be discoveredeverythingto , Driving is a spectacular be obliterated . - Jean Baudri ard 11

Many researchcentershave expendedand continue to expenda significant effort on providing perceptual supportfor roadfollowing. We will surveyonly efforts, one in Europeand the other in the United two of the most successful . States a Dickmannsand Zapp ( 1985 havedeveloped high-speedroad following ) at usinga windowing operates system the Military Universityof Munich. This system to techniqueto enforcefocus of-attentionmechanisms meetreal-time

289

Basisfor BehaviorBasedControl Perceptual

constraintsHigh-speedvehicledynamicshavebeenconsidered . in processing their initial testbed, a paneltruck calledVaMoRS(panel(A ) of figure 7.21). Autonomousroad following has been achievedat speedsup to the vehicle' s limit ( 100km hour) . A dedicated / to microprocessor assigned eachfeature " tracks it in its own window. Anticipatory control is utilized via a " preview ' s modeled window basedon the vehicle . , dynamics Figure 7.22 illustratesthe overall control architecture Limited obstaclerecognitionhas also beenprovided . " (Graefe1990 . An evenfastervehiclp vaMP (panel( B) of figure 7.21), ) further pushes envelope high-speed autonomous the of road following. Additional , research extended methodto autonomous has this landingof an airplane (Dickmanns1992 . ) In the United States CarnegieMellon University has a long track record , in road following autonomous robots Figure 7.23 showsone of their earliest . TERREstrialnaviGATOR), that was used amongother , systems Terregator( , et things, for following sidewalkson campus( Wallace al. 1985 . Initially , the ) sensor usedwasa singleblack-and white videocamera but only environmental it waslater upgraded color ( Wallace al. 1986 . to et ) ' Wallaces ( 1987 subsequent work at CarnegieMellon on the Navlab (figure ) 7.24) demonstrated ability to follow roadsstreaked the with shadows and in . based poorly registered the color spectrum A patternclassificationscheme on pixel valueson a color surfacedistinguish sunnyand shaded es road from nonroadregions The imagepixels associated . with the roadare groupedinto a . evolvedinto singleregionthat is then usedto steerthe vehicle This approach the SCARFsystemdescribed below . We have already discussed two other recent road following methodsdeveloped at CarmegieMellon: Ulysses in section7.4.2, and FERMI, in section , 7.5.7. We will now review several others . . The SCARF systemwas initially developedat CMU by Crisman ( 1991 . ) This road following systemrelieson color imageryto detectroadswith which featurebasedalgorithmshad seriousdifficulties: for example roadswith degraded , . , edgesand surfaces and thosewith significant shadowing No three .road model is constructed(consistentwith the behaviorbased dimensional ) approach while SCARF robustly provides control feedbackregardingroad . In its most recentimplementation(Zeng and Crisman 1995 color position ), are of . categories providedwithin the RGB (red-greenblue) space the imagery Pixels are mappedonto thesecategories received reducingthe 24-bit RGB as , color imageto a 6-bit format. Statisticsarerespectively on gathered thosepixels that correspondto road and nonroadareas and in somecasestheseare , further partitionedinto shad d and nonshaded areas A Gaussian . road model , ~

290

Chapter7

(A)

(B)

7.21 Figure

291

Basisfor Beha or-BasedControl Perceptual ~

(C )
) Figure 7.21 (continued road following autonomous : robots (A ) VaMoRS ( B) vaMP and(C) vaMP , , High-speed vision system (Photographs . . courtesyof ErnstDickmanns) generated from the statistics of previous imagery , is then used to classify the 6bit format into road and nonroad regions, providing the necessary information on road location for steering purposes.

Also at CarnegieMellon, Pomerleau( 1993 developedanotherclassifica ) tion system based neuralnets calledALVINN (Autonomous on Land Vehicle , , in Neural Nets . An imagearray or retina of 30 x 32 pixels servedas the input ) to layer. This was connected four hiddenunits that in turn projectedonto the output layer, quantizingthe steeringinto thirty discreteunits rangingfrom to sharpright to straight ahead sharpleft (figure 7.25). The systemwastested on the Navlab system(panel(A ) of figure 7.24) and later ported for use for road following tasksfor the UGV Demo II project (chapter9). For training purposesa humandriver first takesthe vehicle over the road. , methodtrains the network (section8.4). During driving, the back propagation After training, the systemis capable following the roadtypesandconditions of it encountered . ALVINN hashadsuccessful runsup to 90 mileswithout human

292

Chapter?

SUPERVISORY

and Planning Situation Assessment

-forward MODE and Feed Monitoring Rule SELECTION Selection Programs ------- ~ ------------------ ------- ------------------ ---- --------REFLEXIVE t State Feedback _ - f +, BEHAVIOR Estimation '.,:..1 Control Laws

SENSORS

ACTUATORS

Figure7.22 . Architecture in VaMoRS used alongthelinesof hybrid projectNotethepartitioning in 6. discussed chapter systems

Figure7. 23 -Mp .llnl1 Institute of Robotics , Camegie I Univer . (Photographs courtesyThe Terregator . sity)

293

Perceptual Basis for Behavior -Based Control

interventionon highwaysand has also beensuccess fully usedon single and multilanedirt andpavedroadsof varioustypes . The No HandsAcrossAmerica Navlab 5 USA tour is one of the most ambitious exhibitionsof a~tonomousdriving to date (panel (B) of figure 7.24). The vehicle a modified 1990PontiacTrans , , Sport drove almost 3,000 miles from Pittsburgh Pennsylvaniato SanDiego, California. Autonomous , , , autonomously visual steeringwas usedfor 98.2 percentof the trip at speeds averaging 57 miles per hour. The vision systemusedfor this project developed at , -Mellon, and called RALPH (Pomerleau1995 hasa simplecontrol ), Carnegie : process 1. An imageis acquired . 2. Irrelevant portions of the image are discarded(i .e., a focus of-attention . is usedto constrainthe processed ). data trapezoid 3. The remaining image is subsampled yield a 30 x 32 image array that to includesimportantroadfeatures . -and 4. The road curvatureis then computedusing a generate -test strategy , the rows' low-resolution imagery by a predictedcurvatureuntil it shifting " becomes straightened The curvaturehypothesis ." with the straightest features wins. ' 5. The vehicle s lateraloffsetrelativeto the road' s centerlineis thencomputed whenthe vehicleis centered usingtemplatematching(from a templatecreated in the lane on a one dimensional scanline across road. the ) 6. A steeringcommandis then issued and the processbegins again from , 1. step Only the templatein Step 5 needsto be modified when RALPH encounters new road types This templatecan be createdas needed . during driving without humanintervention using look-aheadand rapid adaptation and swapped , , in automaticallyas needed The highest speedRALPH has achievedon the . Navlab5 is 91 miles per hour on a testtrack.

7.6.2 VISual Tracking

. Evenif you' re on the right track, you' ll get run over if youjust sit there - Will Rogers

decadesInstead . Visual trackinghasbeena heavilyresearched for several area we present threeexamples recent of of attempting comprehensive a , survey just and work thathavebeenfieldedon actualrobotichardware thatexploit, at some . level, the notion of taskdirectedness

294

Chapter?

(A) Figure7. 24

295

Basisfor BehavtorBasedControl Perceptual

( B) ) Figure 7.24 (continued , (A ) Navlab 1 and ( B) Navlab 5. (Photographs courtesy of The Robotics Institute -Mellon University) . Carnegie

Woodfill and Zabih' s researchat Stanford University ( 1991 produceda ) motion-based trackingalgorithmfor keepinga movingpersonin view of a mobile . Correlationalmatching robot by panningthe robot' s cameraas necessary wasperformedon a pixel by pixel basisto computean initial disparitymapthat ' smoothed comparingeachpixel' s valuesto its neighbors wassubsequently by . it andassigning the mostpopularonefor the neighborhoodThis algorithmran -nodeconnectionmachine(a very powerful computer processing on a 16,OOO ) of . 15 pairs of imagesper second Segmentation the moving object from the . motion field wasthen performedusing histogramming techniquesThe tracking ' s location to constrainthe interpretation processalso projectedthe object of the motion field. This systemworked for both indoor and outdoor scenes sufficiently texturedto producea rich motion field. ' , , ) Prokopowicz Swain and Kahn s work at the University of Chicago( 1994 of takesadvantage the wide rangeof well- developed trackingalgorithmsavailable , including a simplified version of the Woodfill and Zabih algorithm described above a correlationbasedtracker and a color histogramtrackerthat , , ' . of takesadvantage the targets color properties Additional trackersbasedon

296

Chapter?
YEA LA UNITS 30 ( OUTPUT

INPUT IMAGE (30x 32 )

HIDDEN UNITS

7.25 Figure ' ALVI N N neural s network architecture .

binocularvision arebeingdevelopedThis approach . focuses understanding on therelationshipbetween algorithms the target andthe environment the within , , which the agentoperatesThreebehavioraltrackingtasksaredefined . : . Watch wherethe robot is stationaryandthe targetis moving. : . Approach wherethe robot is moving andthe targetis stationary : . . Pursue whereboth the robot andtargetaremoving. : , , , (speedlocale appearance By definingthetargetclassproperties changesetc.) for a wide rangeof objects a setof environmental characteristics , background motion, unevenlighting, visual busynessand so forth) and the , ( background conditionsunderwhich the varioustracking algorithmssucceed fail , selection or of the algorithmsat run time basedon agentintention targetproperties , , and environmental conditionsbecomes feasible This approach . explicitly acknowledges theseinterdependencies by doing so, significantly enhances , and ' the systems overallrobustness .

right

~
sharp

ahead

straight

sharp-left

STEERING COMMAND

IN

VIDEO

297

Basisfor Beha ior-BasedControl ~ Perceptual

Figure7.26 -jugglingrobot ( Photograph .) of . Rochester balloon courtesy BrianYamauchi at Anotherinteresting developed theUniversityof Rochester trackingsystem for Yamauchi Nelson1991 wasusedto controla robotic manipulator the and ), ( task of juggling a balloon (figure 7.26). Threeconcurrentparallel behavioral : agentswereimplemented . Rotationaltracker keepsthe robot facing in the balloon' s direction. : . Extensionaltracker keepsthe arm extendedso that it remainsbeneaththe : balloon' s position. . Hitter: providesthe upwardforce to the balloon whenthe balloon is immediately abovethe paddleheld by the arm. The balloons were partially inflated with helium to slow them during fall. runs yielded 7.0 bounceswith a 39.4-second An average 20 experimental of . duration with a maximum run of 20 bounceswith a 104 secondduration balls abounds e.g., Aboaf, with conventional Other work in juggling systems ( Drucker and Atkeson 1989 Buhler, Koditschek and Kindlmann 1989 but it , ; ) , . often employsthe reason predictable physicsof the situation Balloonjuggling ably is inherentlymore complexand unpredictablemaking it more suitable , . for behaviorbasedsolutions

298

Chapter7

7.6. 3 Robotic Heads All I askof my body is that it carry aroundmy head . - ThomasAlva Edison

To many the pinnacleof visual sensing occursin the designanddevelopment , of a robotic head A typical headconsists two video cameraseachof which . of , has severalcontrollable degreesof freedomD OFs). To coordinatesuch a , complexsystem severalbehavioralcontrol subsystems may be used( Brown 1991 : ) . Saccadethe open loop rapid slewing of the camerain a given direction, : often usedto repositiona camera whena targetmovesout of the field of view. . Smoothpursuit: continuoustrackingof a moving target . . Vergence measuringthe disparity betweenthe two camerasfocusedon a : to targetandthen moving oneof the cameras reduceor eliminateit. . Vestibulo -ocularreflex: open loop control usedto stabilizethe headcameras relative to body movement eliminating apparent motion due to translationor , rotationof the robot base . . Platform compensation used to prevent the camerapositioning systems : from reachingtheir limits. When a limit is approached any particularDOF, for the open loop motionrepositions joints of the roboticplatform without moving the cameras themselves . Robotic headscan rangewidely in complexity and cost. At the low end of the price scale we canfind a robotic headthat costslessthan $ 1000(Horswill , andYamamoto1994 . It includestwo low-resolutionCCD cameras a fourand ) DOF active head It has beenusedfor experiments vergence . in and saccadic motion. Ferrier and Clark ( 1993 at Harvardhavedeveloped more complexhead a ) of saccades A ) of figure 7.27). An even , pursuit and vergence , capable (panel( more complicatedsystem(panel( B) of figure 7.27) with thirteen DOF has been developedin Sweden(pahlavanand Eklundh 1993 . Primary reflex es ) ) developedfor this KTH (Kungl TekniskaHogskolan head include smooth .A , pursuit, vergenceand involuntarysaccades large numberof robotic heads havebeendevelopedThe interested . readeris referredto ChristensenBowyer , , andBunke 1993for more information. , Perhaps the most ambitious robotic head (and torso as well) project to date is Cog, a robot constructed Brooks and Stein ( 1994 at the MIT AI ) by with auditoryaswell asvisual (figure 7.28). This robot is equipped Laboratory

299

Basisfor BehaviorBasedControl Perceptual

sensorswith pan tilt , vergenceand saccade ( , ) capabilities and hasin its initial two six-DOF arms and a torso with a three OOF hip and a configuration three DOF neck. Conductiverubbersensors be usedlater to impart touch will most of the sensingto the robot. Cog is intendedultimately to encompass of aspects behaviorbasedvisual perceptionwe havediscussed throughoutthis section saccadessmooth pursuit vergence vestibuloocular reflex, visual : , , , routines and headbody-eye coordination This project confrontsheadon the . , -based issues scalabilityin subsumption of behavioralmethods .

7.7 CHAPTER SUMMARY

. Traditional perceptionhas beenconcernedwith constructingan intention -based free model of the world. Newer approach , generatedfor behavior es ' into . , systemstakethe systems motor requirements accountin their design . These behaviorbasedalgorithms are highly modular providing targeted , . capabilitiesfor specificmotor needsandenvironments . Biological studieshaveprovidedsignificantinsight into the designof these : systems . Affordancesprovide a new way of thinking abouthow to perceivebasedon what environmental . opportunitiesareaffordedthe robotic agent . The dual systems what andwherecanbe associated th hybrid architectural of w~ . design . Commonrobotic sensors include shaftencoders deadreckoning inertial for , ultrasound video, and laser , , , navigationsystems global positioning systems scanners . . Schematheory visual routines perceptualclasses and lightweight vision , , , methodsfor describingperceptual . modules providediffering . Action-orientedperception threemeans which perceptual modules provides by canbe coordinated : . Sensorfission: Individual modulesarededicated eachbehavior to . . Action-orientedsensorfusion: Recursivelydefinedmodulescombinemultiple sources informationinto a singlepercept of . . Sensorfashion(perceptual : Variousperceptual modulesare activated ) sequencing andcoordinated differing points in time and space needed at as . . Active perception enables perceptual the to motor process control supporting . behaviors . Expectations focus of attentioncan be usedto constrainthe perceptual and ' s inherent . process complexity

300

? Chapter

(A) 7.27 Figure

. Significantresultshavebeenachieved the areas high-speed in of roboticroad visual tracking andthe designof complexrobotic heads . , following, . The following list summarizes design principles for perceptualalgorithms the in supportof reactiverobotic systems : . Don' t design one algorithm that does everything rather tailor perception : , to meetmotor requirements . modularly . Closelycoupleperceptionandmotor control. . Action-orientedperception centralto achievingrapid response is . . Exploit expectation whenavailable from previousimages object , ( knowledge models etc.) . , . Use focus of-attentionmechanisms constrainsearch use computational to ; . powerwhereit is mostlikely to yield results . Organizeperceptualstrategiesusing sensorfission fusion, or fashion as , needed .

~0

J PerceptuaBasisfor BehaviorBasedControl

I
5

7.27 (continued ) - ~ . . A ) Harvardhead ( photograph . ( courtesyof Nicola Ferrier) (B) KTH head ( Photograph courtesyof Jan Olof Eklundh.)

~ s

7 Chapter

i ~~

Basisfor Behavior BasedControl Perceptual

(B ) ) Figure7. 28(continued : . of Brooks .) ; Cog (A) full view (B) head( photographs courtesy Rodney

Chapter 8 Adaptive Behavior

It is impossibleto beginto learnthat which one thinks one alreadyknows . - Epictetus . A mind oncestretched a new ideaneverregainsits original dimension . by - Oliver WendellHolmes The reasonable adapts man himself to the world; the unreasonable persists trying man in to adaptthe world to himself. Therefore all progress on the unreasonable man. , depends - George BernardShaw

? ROBOTS LEARN 8.1 WHYSHOULD

. Learningis often viewed as an essential part of an intelligent system Indeed some arguethat without this ability, there cannotbe intelligencepresentat AI all: " Learning is, $ r all , the quintessential issue. . . . I will now give a definition of AI that most of our programswill fail. AI is the scienceof for endowingprogramswith the ability to changethemselves the better as a " result of their own experiences(Schank1987 pp. 63- 64). ,

based

systems

algorithms

behavior

based

for

genetic

used

behavior

capabilities

networks

within

algorithms

neural

learning

learning

have

learning

to

for

learning

of

control

need

types

robots

behavioral

major

opportunities

reinforcement

why

the

the

fuzzy

in

Objectives including , understand recognize understand learning To To To . . . 1

systems

Chapter 2 and

306

Chapter8

But what do we meanby learningor adaptation As with manyother terms ? we haveencounteredthereis no universaldefinition: , " . " Modification of a behavioraltendency experience ( Webster ). 1984 by . "A learningmachine broadlydefined is anydevicewhoseactionsareinfluenced , , " ) by pastexperiences ( Nilsson1965 . . "Any changein a systemthat allows it to performbetterthe second time on " of the sametaskor on anothertaskdrawnfrom the samepopulation repetition (Simon 1983 . ) . "An improvementin information processing ability that resultsfrom infor" mationprocessing ) activity ( Tanimoto1990 . Our operationaldefinition will be: withinan agent overtimeenable to perform that it more Learning produces changes withinits environment . effectively Although this definition will not satisfy all, it providesus with a meansfor metrics againstwhich an agent measuringlearning by defining performance canbe measured before during, andafter learninghasoccurred . , How then can we relate learning and adaptation Adaptationrefers to an ? ' in agents learningby making adjustments order to be more attunedto its environment . Phenotypicadaptation occurswithin an individual agent whereas , . genotypicis geneticallybasedand evolutionary Adaptationcan also be differentiated : on the basisof time scale acclimatizationis a slow process but , homeostasis a rapid, equilibrium-maintainingprocessWe differentiatefour is . from McParland1981 : (adapted ) typesof adaptation ' s individual behaviors . BehavioraladaptationAn agent : are adjustedrelative to oneanother . . EvolutionaryadaptationDescendents overlong time scales based on : change or in . the success failure of their ancestors the environment ' . Sensoradaptation An agents perceptualsystembecomes : more attunedto its environment . . Learningas adaptationEssentiallyanythingelsethat resultsin a moreeco : fit agent . logically in , Adaptationmay producehabituation an eventualdecrease or cessation numeroustimes. This when a stimulus is presented of a behavioralresponse . Sensitiza is responses process usefulfor eliminatingspuriousor unnecessary tion is the opposite an increasein the probability of a behavioralresponse , . when a stimulus is repeatedfrequently Habituation is generally associated

307

Adaptive Behavior conventional feedback

Figure 8.1 . Adaptivecontrol system

, with

, relatively insignificant stimuli suchas loud noise whereassensitization occurswith moredire stimuli like electric shocks . From a controlsperspectivewe canmoreeasilydifferentiateadaptation and , ' . learning Adaptive control, an early 1950sexampleof a systems changing to better fit its environment usesfeedbackto adjust the controller' s internal , ( ) parametersAstrom 1995. Figure 8.1 illustratesan adaptivesystemthat uses feedbackin both the traditional senseand for internal modification of the controlleritself. in , , Learning on the other hand can improveperformance additionalways , by . ,introducingnew knowledge(facts, behaviorsrules) into the system . , . generalizing from multiple examples . concepts . specializingconcepts particularinstances are in someway different for that from the mainstream . . reorganizing informationwithin the systemto be moreefficient. the . creatingor discoveringnew concepts . . creatingexplanations how thingsfunction. of . reusingpastexperiences . Artificial intelligenceresearch devotedconsiderable has effort to determining the mechanisms which a robotic systemcanlearnsomeof thesethings , by , leadingto a wide rangeof learningsystemsincluding

. Reinforcement learning : Rewardsand punishmentsare used to alter /or numericvaluesin a controller . . Neural networks: This form of reinforcementlearning uses specialized architectures which learning occursas the result of alterationsin synaptic in . weights

led ) System

Control Element adjustment . Leaming parametric ( ' Controller

308

Chapter8

. Evolutionary learning: Geneticoperators and suchas crossover mutation , to more efficient control are used over populationsof controllers leading , . strategies . Learning from experience . . Memory -basedlearning: Myriad individual recordsof pastexperiences are for control laws. usedto derivefunction approximators . Casebased learning: Specific experiences organizedand stored as a are case structure then retrieved and adaptedas neededbasedon the current , . situationalcontext . Inductive learning : Specific training examplesare used each in turn, to , . or and concepts controllers generalize / or specialize . Explanation-basedlearning : Specificdomainknowledgeis usedto guide . the learningprocess . Multistrategy learning: Multiple learningmethods competeandcooperate . with eachother eachspecializingin what it doesbest , havebeenexploredto varying degrees Many of the abovelearningmethods . in behaviorbasedrobotic systems In this chapterwe look at how learning methodscan be effectively exploited in behaviorbasedrobots and study a wide rangeof constructed systemsthat provide theseagentswith the ability . within their environments to improvetheir performance 8. 2 OPPORTUNITIES FOR LEARNING IN BEHAVIOR -BASED ROBOTICS
. Learningis not compulsory Neitheris survival. - WEdwards Deming Where can learning occur within a behavior -based robotic control system? To answer this effectively , we need to revisit some of the notation we developed in chapter 3. Recall that a functional mapping .8i defines an individual behavior that acts upon a given stimulus Si to produce a specific response ri . A gain ' value gi is used to modify the response s overall strength as a multiplicative constant. ri = gi * .8i(Si) Within the context of the individual behavior, what can be learned? I . What is a suitable stimulus for a particular response? That is , given a desired ri , what Si is appropriate ?

309

AdaptiveBehavior

2. What is a suitableresponse a given stimulus That is, givenaparticularS for ? for ;, what is r ; ? This identifiesa point behavioralresponse a singlestimulus but doesnot specify.8 in its entirety . ; 3. What is a suitablebe~aviora1 an mappingbetween existingstimulusdomain andrangeof responsesThat is, what is the form of .8 ? ? ; 4. What is the magnitude the response That is, what is the valueofg ; ? of ? 5. What constitutesa whole new behavior for the robot (i.e., new stimuli and or responses / )? Recall further (section 3.4.2) that behaviorsare grouped into assemblages that specify the global response of the robot for a set of behaviorsB with p associated to gainsG anda givensetof stimuliS whensubjected a coordination function C: p = C (G * B (S . ) ( Cisgenerally either competitive(arbitration or cooperative fusion) or some combinationof the two. Within the contextof a behavioralassemblage ? , what canbe learned 1. What setof behaviors the of .Biconstitutes behavioral component an assemblage B? ' 2. What are the relative strengths eachbehaviors response of within the assemblage ? That is, what is G? 3. What is a suitablecoordinationfunction C? Problems oftenaccompany , opportunitiesandlearningis not differentin this . respect . Credit assignment : to problem How is credit or blameassigned a particular ) pieceor piecesof knowledgein a largeknowledgebaseor to the components of a complexsystemresponsible either the success failure of an attempt for or to accomplish task? a . Saliencyproblem What features the availableinput streamarerelevantto : in the learningtask? . New term problem When doesa new representational : consttuct(concept ) needto be created capturesomeuseful featureeffectively? to . Indexing problem How can a memorybe efficiently organizedto provide : effectiveand timely recall to supportlearningandimprovedperformance ? . Utility problem How doesa learningsystemdeterminethat the information : it containsis still relevantanduseful? Whenis it acceptable forget things? to Robots can potentially learn how to behaveby either modifying existing . behaviors(adaptation or by learningnew ones This type of learningcan be )

310

8 Chapter
' relatedto Piagets theory of cognitive development (Piaget 1971 in which ), of assimilationrefersto the modificationor reorganization the existing set of available behaviors schemasandaccommodation the process is involvedwith ( ) the acquisitionof ne~ behaviorsRobotscan alsolearnhow to sense . correctly by eitherlearningwhereto look or detenniningwhat to look for. A robot can also learn about the world' s spatial structure We discussed . this in chapter5 in the context of short term behavioralmemory and long. term memorymaps We do not discussspatialmemoryandlearningfurther in within the behavioral this chapterandfocusinsteadon learningandadaptation control system . As we haveseen robotslearnby widely varied methods that can be classi , fied alongseveral dimensions Tan1991 : ( ) . Numeric or symbolic: Symbolic learning associates with representations within control systemsThesecan result in . the numbersinherentlygenerated suchas logical assertionsproductionrules, and semantic , symbolic structures . of numericquantities Neural . networks Numericmethods learningmanipulate of es. networksand statisticalmethodsareprime examples numericapproach . Inductive or deductive Generalizingas a result of learningfrom examples : . a or experiments typical of inductivelearning Deductivelearningproduces is moreefficient conceptfrom an initial oneoriginally providedto the robot. . Continuous batch Learningcanoccureitherduringthe robot' s interaction or : with the world (continuousand on-line) or insteadthroughits acquisitionof a large body of experience prior to making any changeswithin the behaviors . (batch ) Most of the methodsapplied to behaviorbasedrobotic control systemsto . date are both numeric and inductive Someof thesemethodsare on-line and . continuousand othersare more batchoriented The remainderof this chapter success appliedto revealsmoreaboutvarioustypesof learningparadigms fully . different aspects behavioralcontrol systems of

LEARNING 8.3 REINFORCEMENT

The man who setsout to carry a cat by its tail learnssomethingthat will alwaysbe usefuland which will nevergrow dim or doubtful.
- Mark Twain

Reinforcement learning is one of the most widely used methodsto adapt a . . robotic control system It is numeric inductive andcontinuousIt is motivated , , :" , concept the Law of Effect, which states Applying a by an old psychological

311

AdaptiveBehavior
conventional feedback , ~ Controller ~ , Reinforcement -Critic . -. ..., Control led System --

Figure 8.2 Reinforcement . learningsystem

rewardimmediatelyafter the occurrence a response of increases probability its of reoccurring while providingpunishment afterthe response decrease will , the probability" ( Thorndike1911. This notion of rewardandpunishment reinforces ) the causative behavior Modem psychologylargely disputesthe Law of . Effect asthe major basisfor animallearning but the law still providesa useful , modelfor robotic behavioralmodification . To sendthe necessary reinforcement , signal to the control system we need a componentcapableof evaluatingthe response which we will refer to as , the critic . The critic appliesreinforcement the control systemin light of its to evaluation(figure 8.2). This form of unsupervised learning includesno notion of a particulartarget statethat the systemis trying to achieve in contrastto supervised learning , goal in , in which thereis an explicit notion of correctness termsof an optimal behaviorestablished priori. In reinforcementlearning the feedbackto the a , control providesinformationregardingthe quality of the behavioralresponse . It maybe assimpleasa binary pass or a morecomplexnumericevaluation /fail . Thereis no specificationas to what the correctresponse , only how well the is worked . particularresponse Oneof the major problemsassociated with reinforcement learningis credit . Suppose for example we havea collection of N active behaviors , , assignment ;( .8 8;) generatinga global responsep . The critic evaluatesthe results from executingp and determinesthat a reward or punishmentneedsto be of of , applied perhapsby changingthe components G, the relative strengths eachbehavioralcomponentHow can the critic either increase decrease . or its with the Law of Effect? It is hardto determinedirectly strengthin accordance which of the individual components largely responsible the success is for or failure of a responseWe will seehow each of the actual learning systems .

312

Chapter8

address this particular problem Neural network architecturesa es . , presented in , specialcaseof reinforcement learning are discussed separately the following section . A commonmeans expressing for reinforcement learninginvolvesa decision policy. A robotic agentmay havemanypossibleactionsit cantakein response to a stimulus andthepolicy determines which of the availableactionstherobot , shouldundertake Reinforcement then applied basedon the resultsof that . is decision and the policy is alteredin a mannerconsistentwith the outcome , (reward or punishment. The ultimate goal is to learn an optimal policy that ) chooses bestactionfor everysetof possibleinputs. the The issuesin the designof robotic reinforcementlearning systems can be summarized follows (Krose 1995 : as ) . Which reinforcement ? learningalgorithm shouldwe choose 1Wotypespredominate : . AdaptiveHeuristic Critic (AHC) learning The process learningthe decision : of for action is separate from learning the utility function the critic policy uses for state evaluation(panel (A ) in figure 8.3). Section 8.4.2 discuss es connectionistvariationsof this algorithm typically used for behaviorbased , robotics . . Q-Iearning A singleutility Q-functionis learned evaluate : to both actionsand states(panel(B ) in figure 8.3) ( Watkins Dayan 1992 . Lin ' s ( 1992 study and ) ) comparingQ- and AHC-basedreinforcement learningmethodsfor enhancing ' an agents survivability in a simulateddynamicworld providesevidencethat . , Q-Iearning is the superiormethod for reactiverobotic applications Indeed -learningcurrentlydominates -based behavior robotic reinforcement Q learning es. approach . How do we approximatethe control function most effectively? Shouldwe uselookup tablesor discreteor continuousapproximationsand what aspects , ? of the control statesdo we needto represent . How fast do we needto learn? This is strongly dependent the problem on . domain in which the robot is operating Learning that is too slow may not overheadparticularly if the environmentin be worth the extra computational , which the agentis operatingis also subjectto change . , Although a large body of literature exists on robotic learning much research . thereis often a to datehasbeencarriedout only in simulation Because fully in simulationto constructing large leap from implementingstudiessuccess actual robotic hardware we restrict our discussionprimarily to learning , . that haveactually beenfielded on robots The remainderof this sec systems

313

AdaptiveBehavior STATE INFORMATION "' "

STATE Action INFORMATION

l Utility

? Action

Utility (B)

(A)

Figure 8.3 : . Learningarchitectures(A ) AdaptiveHeuristicCritic ; ( B) Q-learning

. tion offers a representative sampling of a range of tasks that benefit from reinforcementlearning strategiesMany additional simulation studiesof be. havebeenconductedwithin the artificial life community haviorallearning systems . Because book is concerned this with robotics we focus on real world , . systems

8.3.1 Learningto Walk

We studied earlier (section 2.5.3) the problem of coordinatingmultiple leg controllersin a leggedrobotic system Obtaining efficient gaits is a nontrivial . . We have seenalso in section2.5.3 that a neural controller can problem to suchasthe cockroach generate gaitsthat correspond thoseof biological systems . Numericreinforcement can learningmethods alsobe appliedin learning . leggedlocomotionin a behaviorbasedcontrol system Maes and Brooks ( 1990 studiedlearning using Genghis a robot hexapod , ) architecture the controller for , (figure3.6). Theyuseda rule-based subsumption which consistsof thirteen high-level behaviorsusing two sensormodalities for feedback two touch sensors : locatedon the bottom of the robot (fore and aft) to determinewhen the body of the robot hits the floor and a trailing ' wheel to measure forward progress(figure 8.4). Genghiss task was to learn to move forward. Negativefeedbackresultswhen either of its touch sensors makescontactwith the ground and positive feedbackwhen its measurement wheel indicatesthat the robot is moving forward. Thesefeedbackresultsare binary (i.e., the sensorsare either on or oft). High-level behaviorsinclude six swing forward behaviors six swing backwardbehaviors and a horizontal , , balancebehaviorthat correctsall of the legs to producea stablehorizontal position.

314

Chapter8

TRAILING WHEEL

Figure 8.4 Sensors usedin Genghisfor reinforcement . to learning Underbellytouchsensors detect ground contact provide negativefeedbackand the trailing wheel provides positive feedback whenthe robot movesforward.

8.3 .1.1 TheLearningAlgorithm

Reinforcement usedto alter the preconditionlist of the subsumption is behaviors on thecriteria of their being: relevant that is, positivefeedback received is , morefrequentlywhenthe behavioris moreactivethanwhenit is not, andnegative feedbackis not receivedat all when the behavioris activeor is received lessfrequentlywhen the behavioris more activethan not; or reliable, that is, the feedbackresults(either positive or negative are consistentwhen the behavior ) ' is active (i.e., the probability of the behaviors occurrence es approach either0 or 1). Each behavior is modified independently beginning with a minimally restrictive list. Eachmaintainsits own performance record a matrix precondition consistingof the numberof times that feedback(bothpositive and negative ) is on or off and whetheror not the behavioris active or not during that time. The resultsare decayed over time so that only a recenthistory is maintained . The correlationfor positivefeedback a behaviorB is computed for by

where

(8.1 )

315

AdaptiveBehavior

is j = numberof timesthe behavioris activewhenpositivefeedback present ; k = number of times the behavior is not active when positive feedbackis ; present number of times the behavior is active when no positive feedbackis ; present and m = numberof times the behavioris not activewhen no positivefeedbackis . present
1 =

The sameequationis usedfor correlatingnegative feedbacksimply by substituting feedback positivefeedback equation8.1. A valuenear for in negative - 1 for the correlationindicatesthat feedbackis not likely when the behavior is activeand + 1 indicatesthat it is quite likely. ' Thebehaviors relevance determined computingthedifferencebetween is by the correlations betweenpositiveandnegative feedback : corr( P , B ) - corr(N , B ) . (8.2) If this value approach + 2, the behavioris highly likely to be active and is es considered relevant that is, positivefeedback receivedwhenthe behavioris is , . active ' The behaviors reliability is then computed by . I D (max( . jp ' . Ip ), max( . jn ' . In In + + } p + Ip } p + Ip } nn I }nn I , (8.3)

where j and I havethe meaningas beforeand the subscripts and n denote p . positive and negativefeedbackrespectively If this value is close to 1, the behavioris considered . , highly reliable but very unreliableif it is nearO If the behaviorprovesrelevantbut not reliable, a different perceptual condition is monitoredto seeif it is responsible the lack of reliability. If it is, for the behavioralrule' s preconditionlist will be altered New performance . statistics aregathered the robot' s performance whenthe new perceptual correlating conditionis eitherpresent absent A correlationwith feedbackis then computed or . asin equation8.1, by substitutingthe new perceptual condition' s being , ' eitheron or off for the behaviors beingeitheractiveor inactive If the resultant . correlationvalue is near + 1, the preconditionlist of the behavioris modified to require the new perceptualcondition to be on; if the correlation value is near - 1, the requirementthat the new perceptualcondition be off is added . If no strong correlation either way is apparent a new perceptualcondition , . is monitored Theseconditionsare continually reevaluated until a sufficiently is achieved . high reliability

316

Chapter8 During execution , the behaviors are grouped based on common actuator control . A probabilistic selection is made based on their relevance, reliability , and newness regarding its appearance in each group . The behaviors are then activated and feedback obtained. These probabilistic aspectsinsure experimentation to avoid premature convergence to a suboptimal solution .

.1.2 RoboticResults 8.3

The following resultswereproduced usingthis learningalgorithm: . Using solely negativefeedbackwith the balanceand six swing forward behaviors Genghislearnedto adopta stabletripod stance keepingthreelegs , , on the groundat all times (the middle leg from one sideandthe front andrear from the other . ) . A second feedback resultedin , , experimentusingboth positiveand negative 's the tripod gait, in which it alternatelyswingstwo sets Genghis walking using . of threelegsforward asthe robot moves

8.3 Learningto Push .2

and At ffiM , Mahadevan Connell( 1991 usedQ-Iearningto teachabehavior ) basedrobot how to push a box. The robot, Obelix, was built on an RWI . 12 inch-diarneterbase Thereare eight ultrasonicsensorsfour of which look , forward, two each to the left and right. Sonar output is quantizedinto two : ) ) rangesNEAR (from 9- 18in~hes andFAR ( 18- 30 inches. A forward-looking of infrareddetectorhasa binary response four inchesusedto indicatewhenthe . robot is in a BUMP state The current to the drive motors is also monitored to determineif the robot has becomephysically STUCK (the input current : exceedsa threshold. Only 18 bits of sensorinformation are available 16 ) FAR) and two for BUMP and bits from the ultrasonic sensors( NEAR or STUCK. : Motor control outputsare limited to five choices moving forward, turning . left or right 22 degrees or turning more sharply left or right at 45 degrees , 's for any of the approximately The robot learningprobleminvolvesdeciding ,

training . performance

additional

which

at point in

the

to

refers improvement system additional learning any

a in

in result not Convergence does

317

Adaptive .Behavior Reward / Punishment

oowzoooa

RESPONSE
Figure 8.5
Obelix ' s behavioral controller (colony architecture ) .

250,000 perceptualstates which of the five possibleactionswill enableit to , find andpushboxesarounda room efficiently without getting stuck. The behavioral controller based on Connell' s colony architecture(section , 4.5.3), consistsof threebehaviors figure 8.5) : ( . Finder behavior which is intendedto movethe robot towardpossibleboxes . , This behavioris rewardedwheneverthe input vector containsNEAR bits. If the robot movesforward and the forward looking NEAR bits are turned on, a + 3 rewardis given. If any NEAR bits that were alreadyon are turned off , a - 1 punishmentis applied Finder is active only when neither of the other . behaviorsis active . . Pusherbehavioroccursafter BUMP resultsfrom a box find andcontinues at leastuntil the box is wedgedagainstan immovableobject suchasa wall. The , robot' s rewardis + 1 if it continuesto move forward and remainsin BUMP, and its punishmentis - 3 if it stopsbeing in the BUMP state(losesthe box). The pusherbehaviorremainsactive for a short time after BUMP contactis lost so that it may recoverfrom possiblesmall errorsin pushing Boxestend . to rotatewhenpushedby the circular robot if not pusheddirectly throughthe centerof drag making this task consider , ably more difficult than it might first . appear . Unwedgerbehavior removesthe robot when the box becomesno longer ' . pushable The robot s reward is + 1 if the STUCK state goes to 0, or its . punishmentis - 3 if it persistsin the STUCK state This behavioris active whenever STUCK bit is on andpersistsfor a shorttime after it goesoff to the ensuresafeextrication .

318

ChapterS

8.3.2.1 TheLearningAlgorithm
, Q-Iearningprovidesthe ability to learn by determiningwhich behavioralactions are most appropriate a given situation the correctglobal robotic response for , for a given set of stimuliS presented the world. An updaterule p by the and is usedfor the utility function Q(x , a), wherex represents states a the : actions resulting Q(x , a) -+ Q(x , a) + ,8(r + AE (y) - Q(x , a , where ; ,8 is a learningrate parameter
r is die payoff (reward or punishment); A is a parameter, called die discount factor , ranging between 0 and I ; E (y ) is die utility of die state y d1atresults from die action and is computed by E (y ) = max ( Q(y , a for all actions a . Reward actions are also propagated across statesso d1atrewards from similar states can facilitate d1eir learning as well . The issue here is what constitutes a similar state. One approach uses die weighted Hamming distance as die basis for die similarity metric . The IS -bit state representation is compacted to 9 bits , d1en die difference between die number of set bits between different states is computed . Some state characteristics are considered more important dian od1ers in preserving distinctiveness. In particular , BUMP and STUCK each have a weight of five , NEAR sonar has a weight of two , and aU od1erbits have a unit weight . Arbitrarily , two states are considered similar if d1eir weighted Hamming distance is less dian three. ' The utility function is used to modify die robot s behavioral responses as follows : all Q( x , a) to O. Initialize Do Forever Determine current world state 90% of the time choose action else pick random action Execute a Determine reward r Update Q( x , a ) as above ' Update Q( x , a) for all states End Do

(8.4)

s via sensing a that maximizes Q( x , a )

x ' similar

to x

319

Behavior Adaptive Another variantof Q-Iearningthat usesstatisticalclusteringfor the similarity metric insteadof weightedHammingdistances alsobeeninvestigated has . 8.3.2.2 Robotic Results The methodsdescribed were testedon the robot Obelix. It was observed that -Iearningovera randomagentsubstantially . usingQ improvedbox pushing The robot' s performance using Q-learning was also comparedto its performance when controlled by a hand coded hand tunedbehavioralcontroller Mahade . , van and Connell ( 1991 statethat the robot was " fairly successful learning at ) " to find andpushboxesandunwedge from stalledstates (p. 772). The ultimate after learning was said to be " close to or better than the hand performance " ibid. . This work' s . codedagent ( ) importancelies in its empirical demonstration ' of Q-learnings feasibility as a useful approach behaviorbasedrobotic to . learning

8.3 Learningto Shoot .3

Researchers OsakaUniversity (Asadaet at. 1995 haveappliedvision-based at ) reinforcementlearning to the task of shooting a ball into a goal, using Q' . learningas their systems underlyingmethodologyIn this instancethe set of statesfor the utility function Q(x , a) is definedin terms of the input visual mounted therobot. The ball' s locationwithin on imageobtainedfrom a camera the imageis quantizedin termsof position (left, center or right) and distance , /near middle, or small far). The goal' s location is quantizedin termsof / , (large the sametwo qualitiesplus relativeangle(left-, right-, or front-oriented. The ) setof states definedby all allowablecombinations thesesubstates of totals319. Two additionalstateseachexist for when the ball or goal is lost either to the right or left. Thesestatesassistin guiding the robot to move in the correct direction, asopposed moving randomly shouldthe stimulusnot be directly to , within the image . present The robot' s actionsetconsists the inputsfor eachof the two independent of motors that power each wheel Each wheels action subsetconsistsof three . commands forward, stop and back. This yields a total set of nine possible : , actionsfor the robot. The robot continuesin a selected action until its current statechanges . The reward value is set to 1 if the ball reaches goal and 0 otherwise the . The discountfactor A, is set to 0.8. Because the difficulty in associating of , a particularstateaction in the pastwith the rewardgeneratedconvergence is ,

320

Chapter8 very hard to achieve if the robot is started in a random state. This difficulty , referred to as the delayed reinforcement problem , is related to the credit assignment problem discussed earlier in the chapter. The problem is addressed here by having a good teacher who provides an intelligent design for learning experiences. The trainer devises easy situations (such as head-on approaches that are near) to allow the system initially to improve its performance in this constrained situation . As this level of competency is mastered, more difficult and challenging states are progressively introduced . This procedure improves learning perfo~ ance convergence dramatically . Suitable training instance selection is used to facilitate many types of learning . An early example is Win ' ston s ( 1975) ARCH program , which illustrated the power of using proper training sequencesfor achieving convergence in inductive concept formation tasks.

8.3 .1 RoboticResults .3
on The systemwas implemented attacked robot equippedwith a radio/video link to an oftboard real-time imageprocessing systemand Sun control computer . The ball waspaintedred andthe goalbox blue to makefeatureextraction were run on the laboratoryfloor. The actualsuccess . simpler The experiments to rate was about50 percentas compared 70 percentpredictedby simulation ratesis not uncommon studies Sucha differencein actualand predictedsuccess . in of because the significantinaccuracies simulationmodelsandnoisein 60 . the input sensorsLooking at the final action statedata approximately percent , -response . of the stimulus mappingswere correctly created Considering to that the robot had no ability whatsoever get the ball into the goal initially , and supportearlier claims that Q-Iearningis a theseresultsare encouraging . robotic learning methodfor behaviorbased feasible

NETWORKS IN 8.4 LEARNING NEURAL

Although learning in neural networks can be viewed as a form of reinforcement learning , neural networks are sufficiently distinct to warrant a discussion on their own. In chapter 2 we presented the rudiments of neural network systems . In this section, we discuss methods for encoding behavior -based robotic control using neural networks and the adaptation methods that can be used to modify the synaptic weights that encode the means by which the robot can respond.

321

AdaptiveBehavior Hebb ( 1949) developed one of the earliest training algorithms for neural networks . Hebbian learning increases synaptic strength along the neural pathways associated with a stimulus and a correct response, strengthening frequently used paths. Specifically ,. * Wij ( t + 1) = Wij (t ) + 11 OiO , j where: Wij ( t ) and Wij ( t + 1) are the synaptic weights connecting neurons i and j before and after updating , respectively ; 1 is the learning -rate coefficient ; and ] Oi and OJ are the outputs of neurons i and j , respectively. . Perceptrons, introduced in section 2.2.3.2, have also been used for robotic learning . Perceptron learning uses a method different from Hebbian learning for synaptic adjustment . The overall training procedure is as follows : an example from a set of positive and negative . learning experiences 2 . Verify the output of the network as to whether it is correct or incorrect . 3 . If it is incorrect , supply the correct output at the output unit . 4 . Adjust the synaptic weights of the perceptrons in a manner that reduces the error between the observed output and correct output . Until satisfactory performance as manifested by convergence achieved ( i . e . , the network has reached its limit of performance ) or some other stopping condition is met . Various methods are used for updating the synaptic weights in step 4. One such method, the delta rule , is used for perceptrons without hidden layers. It modifies synaptic weights according to the formulad ( Wij ) = 1 * Wij * (tj - OJ , ] ) where: d ( Wij ) is the synaptic adjustment applied to the connection between neurons i and j ; 1 is the learning -rate coefficient ; and ] tj and OJare the correct and incorrect outputs, respectively. ( 8.6) Repeat 1. Present ( 8.5)

322

ChapterS The delta rule strives to minimize the error term (1 - OJ using a gradient ) j descent approach.

is Back-propagation probablythe mostcommonlyusedmethodfor updating version . . synapticweights (SeeWerbos1995for a review) It employsa generalized ron of thedeltarule for usein multilayerpercept networks(a commonform of neural networksusedin robotic control and vision) . Usually the synaptic ' , by weights initial valuesaresetrandomly thenadjusted the following update : are rule astraining instances provided Wjj(t + 1) = Wjj(t ) + ,,8j Oj, where 8j = j ( 1 - j )(tj - OJ for an output node and ; ) . W 8j = 0j ( 1 - 0j ) Ek8k j k for a hiddenlayer node . backwardfrom the output layer The errorsarepropagated (8.7)

8.4.1 Classical Conditioning

that Classicalconditioning initially studiedby Pavlov( 1927 assumes unconditioned , ), an stimuli ( US automaticallygenerate unconditioned ( ) responseUR). The US-UR pair is definedgeneticallyand is appropriateto ensuresurvival ' ' . in the agents environment In Pavlovs studies the sight of food ( US would , ) ' s salivation could be that associations result in a dog ( UR . Pavlov observed ) betweena conditionedstimulus (CS), which has no intrinsic survival developed value and the UR. In the dog' s case when a bell was rung repeatedly , ,

the

local

Each

value

myopia

process

time

objective a condition time commonly , in in state function s ' point analogous point infonnation introduces one the parameters each each objective but is local , At at rate of . improve to taken basis minimal efficient climbing . learning the the size measured Hill on is The . . continue step yields only can extremely the maximized is that to minima is should one as

an

minimize

to

seek

that

methods step learning perfonnance one refer local which ,

computed

to , at long ) is next function that as system the

refers traps for states in descent algorithms can procedure objective which the choose

descent one the by to these result essence is as in in can during fast

whereby

as

found

step

optimization

that

Gradient function policy This

323

AdaptiveBehavior

SENSORS 0 slon CIii' Detectors Finder Range Detector Target

Aversive Unconditioned Stimulus 'lone U dit ncon Response Appetitive Unconditioned Stimulus

Robot Motors

Figure 8.6 . supportingclassicalconditioning Learningarchitecture

was with the sightof food, overtime an association madesothat the ringing of . the bell alonewassufficientto inducesalivation Hebbianlearningcanproduce . classicalconditioning An internationalresearch , ; , group ( VershureKrose and Pfeifer 1992 Verat classical as shureet al. 1995 haslooked ) conditioningmethods a basisfor the . self-organizationof a behaviorbasedrobotic system Insteadof hard wiring betweenstimuli andresponses learningarchitecture the relationships , the (figure to 8.6) permitstheseassociations developover time. In their early simulationstudies( 1992 VershureKrose and Pfeifer divide , , ), the positive US fields into four discreteareasin which the attractive(appetitive : , , ) target may appear ahead behind left, and right. A significant turning is requiredwhenthe targetis consider ably to the left or right, a lesser response of . one(if any) if it is to the front or rear The UR setconsists six possiblecommands : advancereverse turn right 9 degreesturn right 1 degree turn left 9 , , , , serveas negative . \\10 , degreesand turn left 1 degree 1 additionalcollision sensors US, producinga response consistingof a reverseand turning 9 degrees from the directionof the collision. The negative aversive US caninhibit ) ( away over collisionstakesprecedence target thepositiveUS, ensuringthat managing . acquisition The CSsusea rangesensorcapableof producinga distanceprofile of 180 are . in degrees the directionin which the robot is heading The readings divided ' : discretelevelsof resolutionbasedon the robot s heading for the into varying - 30 to + 30, there are twenty units covering 3 forward area ranging from , each for the areato the right, ranging from + 30 to + 60, there are ; degrees each and for the areato the far right, ranging five units covering 6 degrees ;

324

Chapter8

from + 60 to + 90, there are three units covering 10 degrees each Areas to . the left and far left haveresolutionssimilar to thoseof the right and far right, . , respectivelymaking a grandtotal of 36 units of rangedata For the neural net~ ork implementation percept -like linear threshold ron , units with binary activationvaluesareused The synapticweightsareupdated . by the following rule:
~ Wij = where is 77 the learning rate; E the decay rate; " N is the number of units in the CS field : 0;, OJ is the binary output value of units i and j respectively ; and () is the average activity of the US field . The robot ' s task is to learn useful behaviors by associating perceptual stimuli with environmental feedback. The behaviors include avoidance, in which the robot learns not to bump into things , and avoidance, combined with approach to a desired target. Note that the robot has no a priori understanding of how to use the range data to prevent collisions from occurring in the US set: This must be learned from the CS. The simulation studies indicate that successful emergent behavior does occur in a manner consistent with the agent' s goals. The authors likened these results to the development of an adaptive field by the construction of a sensor driven control schema. These learned behaviors are consistent with specialized variants of schema-based navigation , discussed in section 4.4 , in which the agent constructs the required responses through environmental interactions. More recently , this work has been extended to vision -based navigation for an actual mobile robot , NOMAD ( Vershureet al. 1995) . The robot ' s learning task involves sorting colored blocks that conduct electricity , either strongly or . weakly , based on feedback from an aversive or appetitive response Because the robot can obtain the US only when it is in actual contact with the block, its goal is to learn the correct response to the color characteristics associated with each block type . In this case the UR arises from a conductivity sensor and the , CR uses color vision . The synaptic update rule used is 770iO jE ~ ( "OWij) ,

(8.8)

325

AdaptiveBehavior ( - ' Wij ( t + 1) = Wij (t ) + v ( t )ai (t ) ( 71aj t ) EWij (t , where

(8.9)

activationof the valueunits, wherethe valueunits provide v (t ) is the average on constraints the system ; ' ai is the neurons firing rate; neuronj ; and aj is the firing rate of the presynaptic rate parameters potentiationand depression for of 1 and E are the learning ] . , synapticstrength respectively Threedifferent variationswereusedin the study: 1. A value systemis present ( Equation8.9 is usedas is.) This value system . in based relevantsensory on exertsaninfluenceon alterations synapticstrength or . of whetherthey are aversive appetitive inputsindependent 2. The valuesystemis turnedoff by settingv (t ) = 1 in equation8.9. to 3. A model is usedthat corresponds Hebbianlearning with no valueunits , of on and a droppingof the dependency synapticdepression aj (i.e., the last term in equation8.9 becomes]aital (t ) - EWij(t . . 1 In the study the robot success , fully learnedto accomplishthe task. The , learning performancevaried dependingon the methodchosen with method 2 resulting in the best performanceand fastestdevelopmentof conditioned . responses : Othershaveusedclassicalconditioning methods Scutt ( 1994 for example ), -like robot (section 1.2.1) consistingof a neuralnet , taught a Braitenberg of only five neuronsto seeklight . In anotherexample Gaussierand Zrehen , 1994 useda variationof Hebbianlearningto teacha small mobile robot, using ( ) a neuralnetwork, to developa topologicalmap, learningsuitableresponses showedthat learned at different locationswithin its world. Theseresearchers similar to potentialfields could be developed usingclassicalconditioning encodings learnedlandmarkpositions(Gaussier methods capableof representing andZrehen1995 . )

8.4.2 AdaptiveHeuristicCritic Learning

With adaptiveheuristiccritic (AHC) reinforcement , learningmethods a critic . of particular statesthrough reinforcement These learns the utility or value learnedvalues are then used locally to reinforce the selectionof particular actionsfor a given state(panel(A ) in figure 8.3). A research group in Spain AHC reinforcement Gachetet al. 1994 has studiedthe useof connectionist ) (

326

Chapter8

' as a basis for learning the relative strengths(i.e., G) of eachbehaviors response 's within an active assemblage . The study specificgoalswere to learn how to coordinateeffectively a robot equippedwith the following behaviors : attraction two perimeterfollowing behaviors(left and right), free-space , goal attraction avoidingobjects andfollowing a path. The outputfor eachof these , , behaviorsis a vector that the robot sumsbefore execution in a mannervery , -basedmethods similar to that of the schema discussed earlier(section4.4) . The AHC network (figure 8.7) startswith a classificationsystemthat maps the incoming sonardataonto a setof situations either thirty-two or sixty-four , environmentAn output layer . , depending upon the task that reflect the sensed a weight matrix W, called the associative searchelement(ASE), containing . computesthe individual behavioralgains gi for eachbehavior Eachelement as Wki of the weight matrix is updated follows: Wki(t + 1) = Wki(t ) + abi (t )eki(t ), (8.10) wherea is the learningrate and eki(t ) is the eligibility of the weight Wki for reinforcement maintainedandupdated a separate as matrix). ( A separate critic element(ACE) determines reinforcement the adaptive signal t ) to be appliedto the ASE. Its weights V areupdated as bi( independently follows: Vki(t + 1) = Vki(t ) + .Bbi t )Xk(t ), ( (8.11 )

where.Bis a positiveconstantand Xk is the eligibility for the ACE. This partitioning of the reinforcement updatingrulesfrom the actionelementupdating is a characteristic AHC methods general of in . The task for the non-holonomic robot (namedRobuter is to learn the set ) of gain multipliers G for a particular task -environmentThree different missions . are definedfor the robot. The first missioninvolveslearningto explore the environmentsafely The secondinvolveslearninghow to move back and . forth betweenalternatinggoal points safely andthe third is to follow a predetermined , alsowithout collisions. path, In the explorationmission initially the robot movesrandomly within the , world. When a collision occurs negativereinforcementis applied and the , robot is movedback to a position it occupiedN stepsearlier (N = 30 for the simulations 10 for the actual robot because tight quarters. For the other of , ) -orientedmissions negativereinforcementoccursunder two conditions : , goal when a collision is imminent, and when the robot is pointing away from the goal (or next path point) and no obstacles nearbyare blocking its way to the . Testson Robuterin a relativelyconfinedlaboratoryenvironment showed goal

327

AdaptiveBehavior

Weigh1 Matrix
Local Reinforcement Signals from Critic

Input Layer
Output Behavioral Gains

l ~::::Cj Adaptive Search Element

Figure8.7 Connectionist learning AHC et ( ) systemafterGachet aI. 1994. successful . learning for the first and secondmissions Successfulsimulation resultswerereportedfor the third missionaswell. Milian ( 1994 also used AHC learning methodsfor navigation tasks A . ) Nomadicsrobot namedTeseoused temporal differencing methods(Sutton . 1988 as a specific instanceof AHC learning In this work, the robot learns ) efficient for a particular task such as moving from one behavioralpatterns , locationto anotherThe robot learnsto avoidunfruitful paths(e.g., deadends . ) on the way to achievingits navigational . goals Basicreactivereflexes areused as primitive behaviors Similarly, researchers the University of Southern . at California (Fagg et al. 1994 have used temporal differencing as a form of ) connectionist AHC learningto permit a robot to learncollision avoidancewall , . following, andenvironmental exploration

328

8 Chapter A research teamat the Universityof Karlsruhe(fig andBerns 1995 applied ) to AHC learningmethods teachleg coordinationandcontrolto a hexapod robot nameLAURON (figure 8.8) . Here also a critic elementgenerates rewardfor a the currentstateand 3 separate action elementgenerates choiceof how to the : act next. Individual leg actionswerelearned(returnandpowerstrokes aswell ) ascoordinated control betweenthe legs.

8.4.3 Learning New Behaviors Using an AssociativeMemory A research , , groupat theUniversityof Edinburgh(NehrnzowSmithers andMca ; ) Gonigle 1993 NehrnzowandMcGonigle 1994 hasstudiedwaysto increase robot' s behavioralrepertoire(i .e., learn new .Bithrough a connectionistassociative . \\10 of their robots namedAlder and Cairngorm were , , memory 1 ' , equippedwith two whisker sensors one eachon the vehicle s left and right front, usedto signal collisions. The robot was also able to sensewhen it is robot, the EdinburghR2, moving forward. In later work a more sophisticated usedfive bump sensorseight infraredsensors proximity detection andsix for , , resistorsto detectthe presence light . of light-dependent Instinct rules, the first approach taken wereestablished the agent which for , , resultedin various forms of locomotion The motor responses . that became associated with theserules correspond closelyto behaviorsin a priority-based such as the subsumption architecture Examplebehaviorslearnedby . system Alder andCairngorminclude (NehrnzowSmithers andMcGonigle 1993 : , , ) . Rule I - Keepforward motion sensor The robot learnsto moveforward. on: . Add Rule 2- Keepwhiskersstraight The robot learnsto avoidbumpinginto : obstacles . . Add Rule 3- Make whiskersrespondafter four secondsThe robot learns : wall-following behavior . . Add Rule 4 - Make alternatewhiskersrespond The robot learnscorridor: . following behavior Figure 8.9 depictsthe learning architecturefor the EdinburghR2 ( Nehm zow and McGonigle 1994 . As is the casefor their otherrobots the two-layer , ) ' ron novel sensorinformation percept -basedneural network s goal is to associate with recently undertakenactions A novelty detectordetermineswhen . conditionshavechanged , sufficiently to warranta changein action which occurs when either the robot' s headinghas changedsignificantly or there is a data memory significantnetdifferencein theproximity sensor . The associative actson a sensorvector containingthe proximity data (quantizedinto a range

329

AdaptiveBehavior

(A)

(B)
Figure 8.8 robot LAURON and( B) its successor LAURON II . ( photographs (A ) Hexapod courtesy . of KarstenBerns)

330

Chapter8

Two- Layer Percept ron PatternAssociator

Figure 8.9 1Wo layer percept learningarchitecture ron .

between and3) andlight-detectordata(digitized into a normalizedrangebetween 0 0 and255). The outputof this associative into memorypasses an arbiter that selectsthe most appropriate action. Only three actionsare defined swift : left turn, swift right turn, andmoveforward. This two-layer percept (eleveninput, threeoutput nodes in contrastto ron ), the multilayer neural networksalreadydiscussedhas the advantage rapid of , . ron , often with only one positive example Thesepercept networks , learning howevercanlearnonly linearly separable functions( MinskyandPapert1969 , ), . althoughthis hasnot yet provento be a problemfor this type of application are . Scalabilityissues asyet unresolved The synapticweight updaterule usedappears equation8.6. For the R2, in is supervised a human in contrastto the instinct rulesusedearlier . , teaching by The experimenter critical feedback its performance on to the provides directly robot by coveringor uncoveringa light sensoron its top. The robot considers itself to be behavingcorrectly while the sensoris dark (positivefeedback. ) When the sensoris uncovered(negativefeedback the robot tries different ) is actionsuntil it finds a suitableone at which point the sensor coveredagain . , . Clearly a good teacheris essentialfor proper learning in this instance This " " , processof applying externalfeedbackis likened to shaping a processof . behaviormodificationusedfor training animals : This robot haslearnedfour basicbehaviors obstacleavoidancebox pushing , . , wall following, and light seeking Combinationsof thesebehaviorsare also teachablepermitting the robot to learn how to navigatethrougha maze , ,

331

AdaptiveBehavin,

for example by usingavoidancelight seeking andwall following in a learned , , , . order

8.5 GENETIC ALGORITHMS

The geneticsof behaviorhas been well studiedin the context of biological certain useful behavioral . systems Breedsof animals are createdto possess or suchas disposition(e.g., viciousness friendlinessin dogs . It is ) properties of also possibleto usecomputationalanalogies behavioralgeneticsto configure . robotic characteristicsIn this section we first review how certainclasses , of geneticalgorithmsoperate then look at specificallyhow they can be used , can . within robotic systemsWe alsoexaminehow their strengths be combined of with the strengths neuralnetworksto yield hybrid adaptiverobotic control . systems

? 8.5.1 What Are Genetic Algorithms

in methods which a Geneticalgorithms(GAs) form a classof gradientdescent -quality solutionis found by applying a set of biologically inspiredoperators high of to individual points within a search , space yielding bettergenerations solutionsover an evolutionarytimescale(Goldberg1989 . The fitnessof each ) memberof the population(the set of points in the searchspace is computed ) how using an evaluationfunction, called the fitness function, that measures ' well eachindividual performswith respectto the task. The populations best are members rewardedaccordingto their fitness and poorly performingindividuals , . or arepunished deletedfrom the populationentirely Over generations , . the populationimprovesthe quality of its setof solutions Although GAs, and to , gradientdescentmethodsin general are not guaranteed yield an optimal solution they generallyproducehigh-quality solutionswithin reasonable , global . amountsof time for certainproblemspacesAs we will see this includes , robots . behaviorbased for the learningof control strategies Genetic algorithmsusually require specializedknowledgerepresentations ' . ) (encodings to facilitate their operators operation The encodingstypically bit take the form of position dependent strings in which eachbit represents set . a genein the string chromosomeAn initial population(a representative . of bit strings is established somemeans often by randomizationGenetic , ) by

332

Chapter8 Reproduce

~ ~

~ ~

~~ ~~ ~ ~
~

~~ ~ ~~ ~~ ~ ~ ~ ~ ~ ~~~ ~~ ~~ ~ ~ ~ ~~ ~ ~
. .

Crossover

~ ~
._

~ ~

~ ~

~ ~~ ~
.~ . .
Mutate
~

I_

I~ ~ ~ ~

_ II _

I_

_ . ~~

~ ~

~~ ~~ ~ ~ ~ ~
8.10 Figure

~ ~~ ~ ~ ~

The genetic operators reproduction , crossover, and mutation .

are of . operators thenappliedto thebit stringencoding thepopulationmembers The three most frequently used operatorsare reproduction crossover and , , mutation (figure 8.10). . ' Prior to the operators application eachindividual' s fitnessis computedusing , the fitnessfunction. For a behaviorbasedsystem this may involverunning , a robot througha seriesof experimentsusing the encodingof the behavioral , controller represented the particular individual bit string encodingbeing by evaluatedThe fitnessfunctionreturnsa valuecapturingthe robot' s overallperformance . for the setof conditionsbeingtested . , Using the reproductionoperator the fittest individuals are copied exactly and replaceless fit individuals This is done probabilistically usually using . , roulette wheel selection increasingthe likelihood of but not guaranteeing , weighted ' ' the fittest individuals reproduction This operators net effect is an . increasein the ratio of highly fit individuals relative to the numberof poor , performers looselyfollowing the Darwinianprinciple of survivalof the fittest. Crossover involves two individual encodings exchanging information , to through the transfer of somepart of their representation anotherindividual . This process createsnew individuals that mayor may not perform better thanthe parentindividuals Which individualsto crossoverandwhatbit string . to exchange usuallychosen are . in parts randomly The net effect is an increase the overall population . Mutation, a simpleprobabilisticflipping of bit valuesin theencoding affects , an individual only and does not increasethe overall population size This .

333

. Behavior Adaptive
local minima a commonproblem randomeffect providesthe ability to escape , methodsJustasin biological mutations most . with gradientdescent associated , a mutationswill lead to inferior individuals but occasionally morefit one will , the . emerge Because probability of mutationis generallyvery low and copies of the most fit individuals result from reproduction this randomness , permits . that would otherwisebe unattainable high-quality solutionsto emerge resultsin a varying populationover time. The useof thesegeneticoperators havea lower fitnessthantheir parentsbut on average Someindividualscreated , ' the entirepopulations overall fitnessaswell asthat of the bestindividualsimproves . with successive , generationsIf properly designedthe learningsystem on eventuallysettles a setof highly fit, nearoptimal individualswith similar bit ' . he strings The final solution s quality andlengthof time to obtainit depend :avily that on the natureof the problem and the valuesfor the many parameters controlproblems andin particularbehaviorbased controltheGA. Fortunately , , control methods are highly compatiblewith thesemethods as they generally , , set size . havea reasonable parameter

8.5.2 Genetic Algorithms for Learning Behavioral Control GAs, althougha powerful techniquefor developingconttol systems require , to the individual learningmethods somerestrictionson implementation compared . we havealreadyinvestigatedSincethesemethodstypically requirea significant , populationof robotsfor fitnesstesting and the robotsmustbe further tested overmany manygenerationsmuchof the learningin geneticalgorithms , , in conducted simulationoff line. As an evolutionarytimescale is of necessity . is needed it is generallyinfeasibleto conductreal-time learning Simulated , at speeds ordersof magnitude , , learning fortunately cangenerallybe conducted degree . fasterthanreal-world testing Assumingthat a simulationhasa reasonable from of fidelity to the real robot andenvironmentthe conttolparameters , can over many generations then be the fittest simulatedindividual developed . actualrobot for use to transferred the in An examplefrom sucha simulationappears figure 8.11(Ramet al. 1994 . ) -basedbehavioralconttoller is In this particular system GA-Robot a schema , , the . evolvedusing geneticalgorithms An encodingis createdthat represents behaviors(goal attraction obstacleavoidance individual gainsof the component , internal to certain behaviors(obstacle , and noise and additional parameters ) , ). sphereof influence noisepersistence In this work, insteadof using a bit more slowly converging string, an encodingusing floating-point valuesfor . is the gainsandparameters used

334

ChapterS begin Obstacles .Create ; 1* Make a new environment * 1 . Build ; 1* Make a new population * 1 Population for 1 to HUMBER do _GENERATIONS begin for 1 to RUNS _GENERATION do _PER 1* Let Robots try to reach goal * 1 begin for 1 to MAXHUMBER _ _STEPSdo begin Robots .Move; end Obstacles .Recreate ; 1* Update environment * 1 end 1* Prepare next generation * 1 Robots . Reproduce; Robots .Crossover ; Robots .Mutate ; end end Figure 8.11 GA-Robot' s main evolutionaryalgorithm . Fitness for an individual is defined as a function of weighted penalties: = collision _weight * number_of _collisions raw _fitness + time _weight * number_of _steps + distance _weight * distance _traveled By altering the penalty weights for each component of the fitness functions , three different classes of robots are evolved , each specialized for a particular ecological niche (figure 8.12) : . Safe: optimized to avoid hitting obstacles while still attaining the goal . . Fast: optimized to take the least amount of time to attain the goal . . Direct : optimized to take the shortest path (which may be slower becauseof reduced speedsin cluttered areas . ) The different behavior of a single class of evolved robots across differing environments is also in evidence in figure 8.13. Although these robots are optimized to avoid collisions , they can still find relatively direct paths in low clutter environments. As the clutter increases however, the paths begin to , , until the robots find many indirect and slow but safe routes through the diverge obstacle field . Using GAs in this way permits an environment - specific control

(A)

(B )

(C) 8.12 Figure worlds (A) safe(B) fastand C) direct of Final 25 , , ( general paths through %cluttered . robots

. , . . . . . . . . .

' . . . . . . . . . . . " '

. . w . . , ~ . . , . . : .

" " ~ . 0

(A)

(C) Figure8.13 Finalpaths saferobots of % ( throughA) 1%, (B) 10 , and(C) 25% cluttered general worlds .

ii

. . . ~ i . ~ . . ~ . ~ ~ ~ . . = . . 0 uE

337

AdaptiveBehavior system to be evolved if desired, filling targeted ecological niches ( e.g ., fast

robots highlycluttered in worlds. )

8.5. 3 Classifier Systems

Other variationson representational . encodingsare also used One common alternativesttategyis the use of a classifier system(Booker Goldberg and , , Holland 1989 . Here, geneticoperators upon a set of rules encoded bit act ) by . elementof a classifiersystemworks in the mannerof strings The performance a productionsystem Preconditions a setof rulesare checkedto determine : for their applicability giventhecurrentsituationalcontext The preconditions . have fixed-lengthbit encodings with values0, 1, or # (don' t care . The actionsideof ) the rule is also a fixed-lengthencodingwith valuesof 0, 1. Conflict resolution methods usedto selectwhich onefrom a setof potentialrules will be used are . The performance elementthenexecutes selected the rule. Credit assignment is critic modulethat evaluates resultsof the chosen the performedby a separate actions(i.e., the fitness and is used as always to guide learning A learning . , , ) elementcreates new rulesusinggeneticoperators . We sawearlier (section3.3.1) that rule-basedsystems a usefulencoding are -basedrobotic systems GA-basedclassificationsystems methodfor behavior . are a naturalfit . One internationalresearch in group centered Milan hasused thesemethods evolvebehavioralcontrol in a systemcalled ALECSYS (figure to 's 8.14 . Testingon a seriesof small robots(AutonoMouse the researcher ) ), havedemonsttated : both stationaryand moving phototaxis learningto approach , ) light sources(Colombetti and Dorigo 1992 . In simulation they have further demonstrated coordinationof threedifferent primitive behaviorsthe , , approachingchasing and escaping using severaldifferent potentialcoordination e.g., combination suppressionand sequencing The chase , , ). operators( behavior operatesalong the following lines: A sensorencodesin a four-bit . string the location of the object to be chased This particular encodingcan serveas the preconditionfor a rule that has an action encodingconsistingof five bits, the first threeencodingthe direction to move one bit for whetheror , not to move and a one bit flag to notify the behaviorcoordinatorthat an action , is recommended the rule. Thesebit stringsrepresenting rules evolve by the geneticoperatorsdescribed earlier creatingnew rules as necessary , using and deleting uselessones accordingto the results of the critic ' s fitness assessment . ALECSYS has also beenapplied to manipulatorcontrol, learning to coordinatevision (exteroceptive and encoder(proprioceptive sensors to ) )

338

Chapter8

PERFORMANCE ELEMENT SENSORS MESSAGES CLASSIFIER RULE SELECTION CONFLICT RESOLUTION ACTUATORS

CRITIC
Figure 8.14 ALECSYS learningarchitecture .

GENETIC ALGORITHM

producegrossmotion guiding the manipulatorto a target object ( patelet ale 1995 . ) Another important approach combining the power of productionrules and genetic algorithmsis SAMUEL, a systemdevelopedat the Naval Research Laboratoriesand testedon a Nomad robot (Grefenstette and Schultz 1994 . ) The task for this systemis to learn how to avoid obstacles to safelynavigate and to a goal in a clutteredenvironment . using sonarand infrared sensorsAt eachtime step an action is producedconsistingof moving at a linear speed between- 1 and 5 inches per secondand a turning speedof - 40 to + 40 . set of , degrees second The standard of geneticoperators selection crossover per and mutationareemployed Perfonnance the top five individualsimproved . for significantly (from approximately72 percentto 93 percentsuccess ) for a set of fifty individual rule sets evaluatedover twenty trials each and evolved over fifty generationsThe initial rule set was obtainedfrom a set . of humancreatedrules and machinegenerated variations Significant simulation . resultswere also generated behavioralscenarios for , involving evasion , , dogfighting minefieldnavigation andprey tracking(SchultzandGrefenstette 1992 . )
8. 5.4 On -Line Evolution

Thoughmost GAs evolvedifferent individualsover time, it is alsopossibleto permit the continuousevolutionof the control systemduring executionusing . es ) geneticalgorithms Steels( 1994 accomplish on-line adaptation treating by the behavioralcontroller as a populationof concurrentactivebehavioralprocessesThe goal for therobotic agent(in this casea small machineconstructed .

339

AdaptiveBehavinr

from LegoTechnics is to survive this meansfinding adequate ) ; energysources (which are depletable and not getting stuck in obstacletraps while seeking ) themout. An initial populationof behavioralprocess is generated compete es that for actuatorconb , producing changesin speedor turning. Fitnessis "oil evaluated over somehistory window, typically looking aboutone secondinto the past accordingto how well the systemresponds satisfyingits survival , to 's needs During this time interval eachprocess contributionto (impact on) the . robot' s control is logged The fitnessand impact a process . had during its last . creationinvolving GA operators history window guidesreproductionNew process is usedto keepthetotal numberof activeprocess in the system es constant . over initial behavioralconfigurations Significant performanceenhancements havebeenachieved . usingthis approach In this system an individual agenthas the ability to respondto continual , in . assume the behavioral that changes its environmentMost GA methods controller ' is fixed during a particularagents lifetime. This methodpennitscontinuous , adaptation which can potentially provide greaterflexibility in adapting ' to evolving ecologicalnichesand it explicitly recognizes agents needto an to . changein response ongoingenvironmentalchange .
s. sis Evolving Fonn Concurrently with Control

Sims ( 1994 hasdeveloped interestingapproach evolving entire robotic an to ) creatures GAs. Given the questionbeing studied Sim' s work hasof necessity using , beentestedonly in simulation Sims allows for the evolution of not . ' , only the agents controller but alsoits morphology(form). Genotypes encoded as directedgraphsare used to producephenotypicstructuresthat constitute the corresponding three dimensional kinematicsystemsRigid, revolute twist, . , and other joint types are permitted The genotypeencodingdetermines . spherical . points of attachment Sensorsattachedto the independentlyevolved control systeminclude contact joint angle and photosensors neural controller .A , , sensor onto effectoroutputs . maps inputs To evaluatefitness a high-fidelity physical simulation is created Various , . : objectivefunctions are establishedfor swimming and walking, the distance ' traveledby the agents centerof massperunit time; for jumping, the maximum clearance achieved the lowestpart of the agent andfor following, the average ; by of to . success speed approach a light source Figure8.15showsseveral fully evolvedcreatures swimmingandwalking. for selectionis based . , Reproductive uponfitness asis normally the case Crossover on involve combiningcomponents the of operations genotypicencodings

340

Chapter8

(A)
Figure 8.15

. directedgraphratherthan bit string manipulation Mutation is also conducted . randomnodesto the genotype graph by adding . The resultsof thesesimulationstudiesareimpressiveIt is hardto envision , giventhe currentstateof the art, how thesekinds of kinematicvariationscould . be conductedautonomouslyNonethelessit is certainly important to recognize , that behavioralroboticsis limited if one considers only a fixed spectrum to robotic structures control. Natureprovidesthis flexibility to its of physical of creaturesRoboticistswould alsodo well to considertheseaspects morpho . . , logical adaptationat the very leastwithin the designprocess

/ 8.5 Hybrid GeneticNeural .6 Learningand Control

havecombinedthe powerof ne~ controllersandgeneticalSeveralsystems at . gorithms Researchers the Centerfor Neural Engineeringat the University California ( Lewis Fagg andBekey 1994 haveusedGA methods of Southern , , )

(B)

Figure 8.15 (continued ) Evolved creatures (A ) Walking creatures ( B) Evolutionary ancestors the water , of , snake and (C) Swimmingwatersnakes( photographs . , of Karl Sims.) courtesy

342

Chapter8

to evolve the weights for a neural controller for a robotic hexapod named , ratherthan using traditional neurallearning(e.g., back propagation. ) , Rodney Fitnessfunctions were definedfor first learning oscillatory control for each . of the legs and then coordinatingthe oscillationsto produceeffective gaits itself, but surprisinglythe robot preferredto walk The tripod gait manifested backwardrather than forward: Evidently this was more efficient for this particular /neural system . mechanicalstructure Similar resultsfor a hybrid genetic were obtainedat CaseWesternReserveUniversity (Gallagherand Beer 1992 . ) on In other work, a Braitenbergstyle neuralcontroller was implemented a with threeambientlight robot equipped availableKhepera smallcommercially and sensors ( pointedto the floor andeight infraredproximity sensors Mondada Floreano1995 Floreanoand Mondada1996 (figure 8.16). Fitnessfunctions ; ) weredefinedfor variousbehaviors including , . NavigationandobstacleavoidanceFitnessmaximizesmotion and distance : . from obstacles . Homing: Fitnessensures the poweris kept at adequate levelsby adding that a light-seekingbehaviorto guide it to its black rechargingareawhen power low. becomes . Graspingof balls using an addedgripper: Fitnessmaximizesthe numberof . ) objects(balls grippedin an obstaclefree environment Geneticalgorithmsareusedto evolvethe synapticweightsfor the neuralcontroller . . In all three cases the targetedbehavioris learnedto varying degrees , Here also backwardlocomotionis a preferredmethodfor the evolvedmobile individual backedup until it encountered . gripper controller The most successful to aroundandattempted grip it. , somethingthenturned

CONTROL 8.6 FUZZYBEHAVIORAL

Any fool can makea rule And everyfool will mind it. - Henry D. Thoreau

In this section we belatedlyintroducefuzzy behavioralcontrol, a variant of , discreterule-basedencodings section3.3.1). We review first the basicprinciples ( . of fuzzy logic andthen its applicationsspecificto robotic systemsSome . for of aspects learningof fuzzy control arethenpresented reactiverobots

343

AdaptiveBehavior

Figure 8.16 . , Ddada and A. Guignard , Kheperarobot. ( photographcourtesy of E. Franzi F. MO . Photographed Alain Herzog) by

8.6.1 WhatIs Fuzzy Control ?

on Fuzzycontrol systems produceactionsusinga setof fuzzy rulesbased fuzzy logic, which is different from conventionalpredicatelogic. In conventional about the world are either true or false: there is nothing in logic, assertions between Valuessuch as true and false are referredto as crisp, that is, they . haveoneexactmeaning Fuzzylogic givesus a differentperspectiveallowing . , variables takeon valuesdetermined how muchtheybelongto a particular to by function). In fuzzy logic thesevariables are fuzzy set(definedby a membership " referredto as linguistic variables which have noncrispmeanings(e.g., fast, , slow, far, near etc.). Membership functionsmeasure of , numericallythe degree of . similarity an instance a variablehasin its associated fuzzy set A fuzzy logic control system(figure 8.17 consists the following: of ) . Fuzzifier: which maps a set of crisp sensorreadingsonto a collection of . fuzzy input sets . Fuzzyrule base which containsa collectionof IF-THEN rules. :

344

Chapter8

Sensors

Fuzzy Inference Engine

Defuzzifier

Actuators

8.17 Figure
. Fuzzy logic control systemarchitecture . Fuzzy inference engine : which maps fuzzy sets onto other fuzzy sets according to the rulebase and membership functions . . Defuzzifier : which maps a set of fuzzy output sets onto a set of crisp actuator commands.

that may be usefulforbehaviorConsideran exampleof linguistic variables basedrobotics steeringcontrol. It might be useful to instruct the robot to turn : in somedirection, but we may not want the behaviorto specify the value of a turn too crisply (e.g., turn right 16.3 degrees. Insteadit may be desirable ) to havea fuzzy output, suchas turn-hard right, or slightly-right, or don' t turn (and similarly for the left). Membershipfunctions encodingthis information might appearsomewhatas shown along the horizontal axis in figure 8.18. betweenlinguistic classesSuppose . Note the overlapin membership similarly that provide linguistic information such we have obstacledetectionsensors as clear ahead obstaclenearright, obstaclefar-right, and similarly for the , left. ( Examplemembershipfunctions are also shown on the vertical axis in , ) figure 8.18 . Simplefuzzy rules canthenbe created suchas ' t turn. . IF clear ahead THEN don . IF obstaclenearright THEN turn-hard right. -far-right THEN turn-slightly-right. . IF obstacle A fuzzy control systemof this sort would startwith crisp sensor readings(e.g., in numericvaluesfrom ultrasound translatethem into linguistic classes the ); fuzzifier; fire the appropriaterules in the fuzzy inferenceengine generating , a fuzzy output value; then translatetheseinto a crisp turning angle in the to defuzzifier asultimately the motor mustbe commanded turn at a particular , . discreteangle

345

AdaptiveBehavior

eft

eft

-ahead

ht

-right

INPUT OBSTACLE LOCATION

Ha < Left orwar Right Hard left ~ ) ~ ~ ~ ~ ~ X ~ ~ right

OUTPUT HEADING
Figure 8.18 functionsand Fuzzy logic for steeringcontrol, showinginput and output membership fuzzy rules relatingthem.

havemore flexibility than conventional -basedmethods rule Fuzzy systems and permit more robust integrationof sensorimotor commands than conventional . production systems Fuzzy control systemsare now pervasivein consumer : , , products washingmachines camcordersand V CRs, to namea few. Additional introductoryinformationon fuzzy logic canbe found in Koskoand Isaka 1993 .

8.6.2 FuzzyBehaviorBased RoboticSystems

Success of fully fielded systemsshow the advantages behavioral fusion or blendingusing fuzzy logic. We reviewtwo of theseefforts.

346

Chapter8

8.6.2.1 Flakey
At SRI, Saffiotti, Ruspini, and Konolige ( 1993b have designeda reactive ) fuzzy controller for the robot PIakey(figure 6.12). Specificbehaviorsare encoded ascollectionsof fuzzy rules. One suchexamplerule for obstacle avoidance IS : -Closeon-left THEN turn . IF obstacleclose in-front AND NCYf Obstacle sharpleft. , Fuzzy rules can also invoke whole behavioralrule sets providing context for interpretingsensordata Thesemetarules describethe applicability of a . behaviorfor a given situation IF context THEN apply(B;) . For example the : , ; rulesbelow specifywhich behaviorsshouldbe activedepending whethera on

collision imminent is . -oft) . IF collisiondanger THENapply (keep . IF NOTcollisiondangerTHENapplyfollow) ( ( )

Remember that the appliedbehaviorsthemselves fuzzy, so dependingon are function structure eachmaybe activeto varyingdegreesthus the membership , , their responses . Thebehaviors implemented schemas are as , smoothly blending eachconsistingof threecomponents : ' . Contextdetermines particularbehaviors relevancy a given situation a to . . A desirabilityfunctionis implemented a setof rulesspecifyingthe control as . regime . A descriptorset definesthe objectsthat must be perceivedor actedupon during execution(e.g., placesandthings in the world). , Using fuzzy control, Flakeycan pursuemultiple goals blendingbehaviors . using rules without requiring strict arbitration Fuzzy control also permitsintegration of a deliberative ( plannerto yield a hybrid architecture Saffiotti, Rus , , ) pini, and Konologe 1993a. Using this controller Flakey deployedprimarily at in anoffice environmentsuccess competed the first AAAI mobilerobot , fully , competitionin SanJose winning secondplacein a competitionemphasizing . the ability to navigatein an obstaclestrewnenvironment

8.6.2.2 MARGE
at Another robot using fuzzy logic, MARGE (figure 8.19 developed North ), CarolinaStateUniversity (GoodridgeandLuo 1994 wasa winner in the following ), ' s AAAI robot . competition MARGE usedfuzzy logic differently year

347

AdaptiveBehavior

8.19 Figure IEEE et . @ . (Reprinted pennission Janet at 1996 1996 .) with from MARGE

348

ChapterS howeverInsteadof allowing contextto enableanddisablebehavioral . rule sets , MAR GE' s controller usesa networkedcollection of distributedfuzzy agents , aUindependent concurrent Implemented and . fuzzy behavioralagentsinclude , , goal seeking obstacl avoidancewall following, and docking. Fuzzy behavioral ~ fusion is conducted using additional fuzzy controllers as multiplexers , to adjust the gains (g;) for eachbehavior Weightedvector summationis the . methodfor producingthe final defuzzifiedcommandsignal A finite statemachine . ' between sequences fuzzy controllerssuitablefor the competitions tasks , suchasoffice rearrangement , which requiredmoving boxesfrom one location to anotherin a clutteredworld. MARGE won first placein this event .

8.6.3 Learning Fuzzy Rules

for robotshaspredominantly Learningin fuzzy control systems behaviorbased focusedon learningthe fuzzy rules themselvesIn work conducted the Oak . at , Ridge National Laboratories Pin and Watanabe 1995 provide one example ( ) for automaticallygenerating fuzzy rule basefrom a userprovidedqualitative a . , descriptionof behavior Learningis vieweddifferently in this system which is ' the ability to reflect the user s intentionsmore effectively A traditional . given rule-based , ), learningsystem TEIRESIAS(Davis 1982 that waslayeredon top of productionexpertsystems serveda similar purpose to facilitate the transfer : of knowledgeinto a usableform and assistin the development a rule base of . In this fuzzy approach the rule baseis generated the following , automaticallyby : process 1. The user entersthe rule strategyfor reactingto a given stimuli (the base behavior in a qualitativeform usinga template . ) 2. The user definesthe input membership functionsfor the stimulus specifically for eachbehavior . 3. The systemcreates skeletonrule basefor this informationand verifiesits a - of . completeness regardingcoverage the stimulus aDdresponse space Output functionsare initially setto a standard value . membership 4. Specializedmetarulesare then generated suppress inhibit behaviors to or sense. The membership functions of the rules are automatically (in the subsumption ) to . adjusted reflectthe desireddominance relationships Successful resultshavebeenachieved usingautomatically generated fuzzy rule bases both indoor andoutdoorrobotic systems for . In a systemmorecloselyrelatedto the othertypesof learningdiscussed earlier in this chapter work at the Laboratoired' lnformatique Fondamentale et , d' IntelligenceArtificielle (LIFIA ) in France(Reignier 1995 has usedsuper )

349

AdaptiveBehavior

vised incrementalmethodsfor learningfuzzy rules. The task hereis to find a behavior . the collectionof fuzzy rulesthat captures robot' s existingsupervised . in Temporaldifferencelearningmethods(discussed section8.4.2) areused In mod , , (parameter particular the systemi ~ capableof rule creation adaptation ification of THEN part of rule), and generalization modificationof IF part). ( Positive reinforcementresults when the robot reachesthe goal, negativereinforcement . when it bumpsinto something Learning occurswhile the robot movesthroughthe world on the way to the goal. At this writing only preliminary this simulationresultswere available but nonetheless work showsmore , ' traditional reinforcementlearningmethods extensibility to more unusualbehavioral control regimessuchasfuzzy logic. 8.7 OTHER TYPES OF LEARNING Severalother methodsfor learninghavebeenappliedto or havepotentialfor . applicationin behaviorbasedsystemsA brief surveyfollows. 8.7.1 CaseBasedLearning
Case- based learning methods use the results of past experiences to guide future . action ( Kolodner 1994) . Experiences are stored as structured cases The basic for case-based learning and acting is as follows : algorithm 1. Classify the current problem . 2. Use the resulting problem description to retrieve similar case( s) from case memory . ' ' 3. Adapt the old case s solution to the new situation s specifics. 4. Apply the new solution and evaluate the results. 5. Learn by storing the new case and its results. At Georgia Tech, Ram et al. ( 1997) have applied these methods in simulation only to a schema based behavioral controller called ACBARR . Cases comprise three components: a set of gains G and several internal parameters used for wandering and obstacle avoidance that represent a particular behavioral assemblage ; environmental information indicating when this configuration was in use; and some local bookkeeping information . The goal becomes learning ' which situations should be associated with which case s behavioral configurations . Figure 8.20 shows the overall system architecture. The system begins with a particular configuration determined by the parameters of the behaviors . When perfonnance inadequacies are determined by such criteria as not

350

8 Chapter
towardthe goal or not moving sufficiently a new casemore making progress , for . of appropriate the task is selectedOn-line adaptation the caseoccursby a methodreferredto aslearning momentum , (Clark Arkin , andRam 1992. This ) method succinctlys. ummarized states that if the systemis doing well, do the , , samething a little more strongly if doing poorly, alter the behavioralcomposition ; to improve its performanceFor example if the systemmovesfrom . , a relatively obstaclefree areato a more clutteredone, obstacleavoidance begins to increase goal attractionto decrease and until satisfactory performance is achieved . Whenever ACBARR encounters sufficientlynovel environment significantly a or modifiesthe original caseretrieved that informationis storedfor future , referencei.e., it learnsto usethosebehaviors nexttime it encounters similar the a , situation The systemis capableof escaping . box canyonsand navigating , complexmazes usingthesemethodswhich a purely reactivesystemwould not normally be ableto do (figure 8.21).

8.7.2 Memorybased Learning

Memory-basedlearning can perhapsbe viewed as casebasedlearning taken to the extreme in which explicit numerical details of every experienceare , remembered and stored Although it has not yet been applied in behavioral . controllers it has been proven an effective techniquein robots for learning , functional control law approximators tasks such as pole balancing juggling for , , and billiards (Atkeson Moore, and Schaal1997 Moore, Atkeson and , ; , Schaal1995 . Simply speaking in memorybasedlearning complex control ) , , functions are approximated the interpolation of locally related past successful by . are experiencesTheselazy learning methods well-suitedfor complex domainswith largeamounts data of .

8.7.3 ExplanationBased Learning

use Explanationbased learning( EBL) methods models(typically symbolic of ) the domainto guide the generalization specialization a conceptby induction and of . Learningoccurson an instanceby-instancebasis with refinementof , the underlyingmodeloccurringat all stepsin the processguidedby an underlying , modelor theory (explanation the world). Domain specificknowledge of is crucial for this process operate to , effectively contraryto the numericmethods of reinforcement neurallearningwe discussed and earlier The robot generates . a plan of action basedon its goal, currentperceptionsand underlying ,

'"# ,e 0. ~ 8 ~ e n . ~ # ~ # ~ ~ ~ ~ P
( Q or

352

Chapter8
032 . 0 c InFluence Goal oF To 5 48 . 0 Sphere Gain Motion 50 . 1 40 . 4 ObJect 00 . Gain 1 Obstacles 58 Goal . 38 510 . 0 50 Gain . Dist 0 0 Noise Direction 39 Contacts 2 . 0 Persistence 260 Noise

(A) 8.21 Figure Effects Adaptation BoxCanyon of on Perfonnance Purely : (A) reactive learning or , no -based used and ; (B Nocases , onlyonlineadaptation ) Case ) ; (C adaptation reasoning onlineadaptation (ACBARR ).

Steps Maenitude

044

614

89

57

15

Influence -

of -

Sphere

Gain -

ObJect

Gain -

Goal

Gain -

353

AdaptiveBehavior

) Figure8.21(continued
Direction 00 . 1 1 Noise Persistence -

(B)
Malnitude Noise

072

990

Goal

To

00

Motion

00

Obstacles

94

Dist

65

00

45

Influence -

of -

Sphere

Gain -

ObJect

Gain -

Goal

Gain -

354

Chapter

) Figure 8. 21(continued
2 case Current Direction 00 . 1 Malnitude 20 Contacts 120 Steps 1 Noise Persistence Noise

355

AdaptiveBehavior

' . or theory of the world it inhabits Basedon the plan s success failure, future arechosen guidedby the theory as . plans of Learning sequences operationsin manipulationtasks seemsto be the domain of choice for EBL scientistsstudying robotics (Segreand Delong 1985 . Other research the manipulationdomain has demonstrated the in that ) in EBL can also be learned(ChristiansenMason , , underlyingtheory required and Mitchell 1991 . Here not only is the plan selectionprocessaffected but , ) , the underlyingtheoreticalexplanation actionis learnedaswell. of Thrun ( 1995 hasdemonstrated ) , hybrid EBL, Q-learning andneurallearning for the navigationof the mobile robot XAVIER (figure 5.15 whoseparticular ), task here is to recognizeand then move to a specific target (green soda can) using sonar vision, and laserstripe rangedata Navigationconsistsof a . , of , . sequence specifiedactions ratherthanbeingcontrolled by a setof activebehaviors . Seven actionsarepermissible including sharpturns, moving forward, , and a specialized hard coded obstacleavoidanceroutine should something , get in the way. A learningepisodeconsistsof the robot startingat somepoint within the lab andterminatingwhen either the robot is directly in front of the the ) targetsodacan(rewarded or the targetleaves field of view (penalized. Q) is usedto determinethe action policies. The domain theory instead , learning of being symbolically represented rules, which is usually the casein EBL, by is capturedin a neural network in advanceusing back propagationtraining. 's The training set for Thrun experiments includes 3,000 instancesfrom 700 . Theseneural networksare usedto predict (explain the ) navigationepisodes reinforcement would resultfrom the applicationof a particularaction The that . first by an ex post facto explanationusing the domain learningprocessproceeds ' thenoccurs theoryof thecurrenttrainingexamples result. Generalization basedon the explanation the training instancein accordance of with the existing . weight spacederivedfrom previousexamplesFinally, refinementoccurs the errorbetween training exampleandthe synapticweights the by minimizing in the networks The net effect in Thrun' s trials was successful relatively . and ) rapid on-line learning(lessthanten minutes andnavigationfor this particular task. EBL hasyet to be extended behaviorbasedsystemsbut thereappears to , to be significantpotentialfor its use basedupontheseotherresults . ,

8.8 CHAPTER SUMMARY

. Robots need to learn in order to adapt effectively to a changing and dynamic environment . . Behavior -based robots can learn in a variety of ways:

356

ChapterS
. They canlearnentire new behaviors . . They canlearnmoreeffectiveresponses . . They can learn to associate more appropriate broaderstimuli with a particular or . response . They canlearnnew combinations behaviors(assemblages of ). . They canlearnmoreeffectivecoordinationof existingbehaviors . . Learningcaneitherbe continuousandon-line or be conducted the end of at an episode manyepisodes or . . Reinforcementlearning is a battery of numerical techniquesthat can be behaviorbased robots : effectivelyusedin adaptive . Using statisticalcorrelationto associate rewardswith actions . . Adaptive heuristic critic methods in which the decisionpolicy is learned , from the utility cost function for stateevaluation TheseAHC . independently methods often areimplemented neuralnetworksystems in . . Q-Iearningin which actionsand states evaluated are . together . Neural networks a form of reinforcementlearning use specialized multinode , , , architecturesLearningoccursthroughtheadjustment synapticweights . of suchasHebb' s rule or back propagation . by an error minimizationprocedure . Classicalconditioning in which a conditionedstimulus is eventually over , time with suitabletraining, associated with an unconditioned , responsecanbe manifested robotic systems well. in as . Simpleassociative memories as can implemented two-layerperceptrons produce for simpletasks . rapid learning . Geneticalgorithmsoperate oversetsof individualsovermultiple generations suchasselection crossover mutation and . , using operators . Effective fitnessfunctionsmust be definedfor the particulartask and environment for successful . , evolutionarylearning By suitableselection particular of ecologicalnichescanbe definedfor variousbehavioralclasses robots(e.g., safe fast, etc.) , . A classifiersystemusesfixed-lengthbit stringrule-based for representations for . discretebehavioralencodings usewith geneticoperators . Evolutionary strategies havebeenusedfor on-line adaptationand changes in physical structurein addition to the more commonapplicationto off-line . learningof control systemparameters . Fuzzy control usesrule-basedmethodsthat involve taking crisp sensorinputs , fuzzifying them conductingfuzzy inference andthen producinga crisp , , . response . Membershipfunctionsmap the inputs onto the degreeof membership a for variable . particularlinguistic

357

AdaptiveBehavior

. Learning can be accomplished fuzzy behavior basedrobot systemsby in intentionsthroughthe automaticgeneration refinement and capturingdesigner of suitablefuzzy rulesfor an applicationor by adapting rulesdirectly using the reinforcement . learningmethods . Many other powerfull "earning methodshavejust begunto be exploredin the contextof robotics including memorybasedcasebased andexplanation , , , basedlearning . . Behaviora1learning haveenabledrobots to learn to walk to push , systems boxes to shoota ball into a goal, andto navigatesafelytowarda goal, among , otherthings.

Chapter9
-

Social Behavior

. The mob hasmanyhead~ but no brains - Englishproverb . A teameffort is a lot of peopledoing what I say
- Michael WInner

Chaptercomplexities Objectives robotic benefits I.To to anddimensions understand along multiagent of able differentteams characterize . 2 bethethe of which .Toorganized systems candifferences and be . communication robots in ,perception .To the behaviorrobotic 3 recognize compared social , with whenlearning associated solitary adaptation to . agents
? THANONE BETTER 9.1 ARETWO(ORN) ROBOTS
When is it better to go it alone, and when to have teammates? This question applies not only to human endeavorsbut robotics as well . As expected, teaming robots together has both an upside and a downside. The positive aspects: . Improved system performance : Where tasks are naturally decomposable, the " divide and conquer" strategy is wholly appropriate. By exploiting the parallelism inherent in teaming , tasks can be completed consider ably more efficiently overall for a wide range of tasks and environments using groups of robots working together. . Task enablement: The ability to do certain tasks that would be impossible for a single robot.

360

Chapter9

. Distributed sensing Information sharingbeyond the range of an existing : sensorsuiteon an individual robot. . Distributedaction at a distance A robot teamcan simultaneously : carry out actionsat manydifferent locations . . Fault tolerance Agent redundancy : and reducedindividual complexity can increase overall systemreliability. The negative : aspects . Interference The old adage"Too many cooksspoil the broth" pretty much : sumsit up. The fact that actualrobotshavephysicalsize providesthe opportunity for blockageor robot-robot collisions The volume of the agentsthemselves . resultsin anoverallreductionof navigational free space whenmorethan onerobot is used This is especiallysignificantin tight quarters . . . Communication androbustness cost : Communication not free. It generally is . , requiresadditionalhardware computational , processingandenergy Communication canalsosufferbecause noisychannelselectroniccountermeasures of , , anddeceitby otheragentscomplicatingreliability. , ' . Uncertaintyconcerningother robots intentions Coordinationgenerallyrequires : . knowing what the other agentis doing, at leastto someextent When this is unclearbecause lack of knowledgeor poor communication robots of , ratherthancooperate . may compete . Overall systemcost: In somecases two robotsmay cost more than one. If , the team can be designedusing simpler lesscomplex robots than would be , , the . requiredindividually, this is not necessarily case In light of the potentially significant advantages afforded by multirobot teamsdespitethe potentialdrawbacksresearchers investigatingcooperative are , societiesbringing a wide rangeof perspectives bearon socialbehavior to : , . Ethological Studyinghow animalscooperate communicate Arkin and : and ( Hobbs 1992 . ) . OrganizationalLooking at how humanorganizations structured Carley : are ( 1995 . ) . Computational models Drawing from computerscience the areas multiprocessing : in of andparallel systemdesign( Wang 1995 . ) . Distributedartificial intelligence Dealing with the problemsof agencyand : , , cooperationusing negotiation deception and methodsfor communication ( Lesser1995. ) . Motion planning Addressing geometricandkinematicproblemsof multiple : the aboutin space Latombe1991. ( ) objectsmoving

361

Social Behavior

. Artificial life: Studying the relationshipsthe multiagentteamsform with of their environmentstypically including aspects competitionas well as cooperation , ) (Langton 1995 . , Independentof the ~ rspective taken many potentially useful jobs for . robotic societieshave been identified Someof the most commonly studied include tasksfor multiagentrobotic systems . Foraging whererandomlyplaceditems aredistributedthroughoutthe environment , ' , andthe teams taskis to carry thembackto a centrallocation. . Consuming which requiresthe robots to perform work on the desiredobjects , . in place ratherthancarryingthembackto a homebase This may involve , or disassembly , suchasin a land mine field. assembly operations . Grazing which requiresa robot team to cover an environmental areaade , ' . quately The potentialapplicationsof this social behaviorinclude lawn mowing lance operationsfor searchand rescue and cleaning operations , , surveil . suchasvacuuming . Formationsor flocking, which require the team of robots to assume age ometric pattern (approximatein the caseof flocking, specific in the caseof fonnations and maintainit while moving aboutthe world. Early work in this ) with theoreticaland simulatedresults(e.g., Sugihara areahasbeenconcerned and Suzuki 1990 Parker1992 ChenandLob 1994 . Behaviorbasedmethods ; ; ) havealso had a significantimpact for coordinatingmultiple graphicalagents within the computeranimationcommunity(Reynolds1987 HodginsandBro; both formation (Balch and Arkin 1994 . Real robots havedemonstrated ) gan . 1995 andflocking (Mataric 1993a behaviors ) ) . Objecttransport which probablycanbe viewedasa subtask certaintypes of , of foraging typically requiresthe distribution of severalrobots around the , . desiredobjectwith the goalbeingto moveit to a particularlocation Particular includebox pushing(Kube andZhang 1992 andcoordinated ) pallet examples and ) lifting andtransport(Johnson Bay 1995 . in Scientistsand engineers Japanwere amongthe first to study coordinated . mobilemultirobot systemsAn earlycellularrobotic (CEBar ) system Fukuda ( et al. 1989 involving the docking of severalsmall robot units to producea ), that larger robot, illustratedcommunicationmechanisms can be usedto support includedinfrared devices . behavior Interrobotcommunication coordinated that usedfor messaging providedpositionalinformationregarding photodiodes has dock location. The CEBOTprogramresearch continuedoverthe years resulting , in an architecture Cal et al. 1995 depictedin figure 9.1, that consists ), (

362
INPUT DEVICES

Chapter9

OUTPUT DEVICES

Figure9.1 CE T MarkV control Ba . system

of multiple parallel behaviors using vector summation as the basis forbehav loral integration ( section 3.4.3.2) . Small teams of mobile robots (figure 9.2) were success fully tested for multirobot goal - oriented navigation among obstacles . These agents use a behavioral suite consisting of go-to - goal , avoid obstacles (using infrared and ultrasonic sensors and an avoid -robot -collision ), behavior that produces a right turn whenever another robot is within a certain distance. At the University of Tsukuba, Premvuti and Yuta ( 1995) experimented with a multiagent robotic system using Yamabico robots equipped with ultrasound , dead reckoning for position estimation , and a communication network capable of transmitting position infonnation . This work focused on cooperation between robots to avoid collisions as they moved about corridors and through intersections.

9.2 ETHOLOGICAL NSm ERA I ON CO T S

As hasbeenour practicethroughoutthis book, we look towardbiological systems feasible to provide insightsregardingthe designof behavior , whenever , basedrobotic systems Ethological studiesshowclearly that multiagentsocieties . offer significant advantages the achievement community tasks A in of . wide range of animal social structuresexists to supportagentagentinteractions . For example urn-level organizations found in schoolingfish, hierar are , chical systems found in baboonsocietiesandcastesystems typified by are are , between theseagents often ) manyinsectcolonies(e.g., bees. The relationships

363

SocialBehavior

Figure9.2 CEBOT MarkV robotteam(Reprinted pennission Calet al. 1995@1995 . with from . IEEE) . determine natureandtype of communication the essential the socialsystem for to succeedThe converse . alsoholds in that the communication abilities somewhat determinethe mosteffectivesocialorganizations a particularclassof for . agents ' ) Tinbergens ( 1953 influential work on animal behaviordescribesa broad rangeof socialactivity: . Simple socialbehaviors . Symp thetic induction, or doing the samethings as others(e.g., the yawn~ ) yawn response . Reciprocal behavior suchascoital activity or feedingyoungthroughinduced , regurgitation . Antagonisticbehavioror simpleconflict . Mating behaviors . Persuasion appeasement and . Orientationor approach . Family andgrouplife behaviors . Flocking and herding defenserelatedbehaviors such as communalattack , individual), and ), (mobs warning (the flock is as alert as the most observant (reducingvulnerability by confusingpredators ) crowding . Congregationusing smell or vision ,

364

Chapter9

. Infectious behaviors spread that the as , , , throughout societysuch alarmsleep

andeating . Fighting behaviors . Reproductive rivals from being at samelocation fighting: Preventing . Mutual hostility: Spreading societyover a region the . Peck order: Establishinga dominancehierarchy and ultimately reducing fighting is . Oneof the mostcommonlystudiedsocialbiological systems that of ants Excellentreferences their socialorganization communication on and methods . include Holldobler and Wilson 1990 and Goetsch1957 Ants typically use communication conveyinformationto one another We haveseen to . chemical earlier (section2.5.1) an exampleof a robotic systemcapableof a primitive ' . methodsForagingmechanisms form of chemotaxis inspiredby ants communication are consider more sophisticated ant colonies however than in in , , ably . robotic systemthusfar developedForagingantslay down chemicaltrails, any dramaticallyincreasingthe efficiencyof foraging while avoidingthe needfor . explicit memoryin the organism Decisionmakingis a collectiveeffort rather and thana masterslavedecision( Deneubourg Goss1989 . This patternis consistent ) with the goal of avoiding hierarchicaldecisionsin a behaviorbased havebeen . robotic society Different foraging patternsfor different ant species of simulated exploiting their tendencies collecting different sizedfood particles , , amongother characteristicsGosset aI. 1990 . One study (Franks 1986 ) ) ( haslookedin particularat the behaviorof army antsin the contextof groupretrieval of prey evaluatingthe relationshipof the massof retrievedobjectsand , the velocity of their return. : A samplingof other interestingsocialethologicalstudiesincludes . the impactof environmental factorssuchasfood supply hunger danger and , , , ) competitionon the foragingbehaviorof fish (Croy andHughes1991 . . mob behaviorand communicationin the whip-tail wallaby illustrating the , that and of multiple agents the natureof communication emergent organization this groupbehavior(Kaufmann1974 . ) supports . primatestudiesregardingthe organization colonies(Altmann 1974 relative of ) . to their environment . the role of display behaviorfor parsimonious communicationmechanisms e.g., Moynihan 1970 . ) ( ' of (Seealso section2.4 for additionaldiscussion ethologicalstudies influence robotics) . on behaviorbased

365

SocialBehavior

9.3 CHARACTERIZATION OFSOCIAL BEHAVIOR

. Designinga societyof robotsinvolvesmanydifferent considerationsA team of agents(either animal or robotic) can be characterized along a numberof dimensions including reliability, organization communication spatialdistribution , , , . , andcongregation

9.3.1 Reliability
Systemreliability is definedasthe probability that the systemcanact correctly in a given situation over time. Parallelism in general increases , , reliability, eliminating the potential for single point failures that would be found in serially structured . on ), systemsHolldobler andWilson ( 1990 based their studies of ants arguethat redundancy within an organization shouldoccur at low levels , ratherthan high levels Redundancy the agentlevel itself, asopposed . at to redundant teamsof agents is considered more important The agentsmust be . , in predisposed somemannerto work togetheras well. This may be something as simpleasnot interferingwith eachotherby stayingout of eachother' s way or as complex as the development a complex vocabularyfor exchanging of . messages

9.3.2 SocialOrganization
Animal societiesare very diverse Wilson ( 1975 established " qualities . ten ) ": of sociality group size demo , , , amountand graphicdistribution cohesiveness , permeability compartmentalization , , differentiation patternof connectedness of roles, integrationof behavior informationflow, andfraction of time devoted , to social behavior Deegener . definedover forty categories animal societies of Allee 1978 . A few distinctexamples socialorganization of includemultilevel ( ) -level structurese.g., hierarchical structurese.g., antcastesystems, flat single ( ) ( , ), schoolingfish), dynamic loosely structuredmobs(e.g., whip-tail wallabies and dominancesystemsor peck orders(e.g., roosting place competitionfor fowl ). The numberand types of constitutentagentsultimately determinethe ' . ) societys performance Specialization(heterogeneity should be basedupon . societalneed One heuristicstatesthat if an eventoccursregularly within the ' societys lifetime, a particular class of agentsshould be presentto handleit societiesshouldbe , ), ( Wilson1975 implying, for robotics that heterogeneous skills. For example it might be if , developed thereis a demandfor specialized betterto createrobotsthat areexpertsat gatheringmaterialandthendelivering

366

9 Chapter
Table 9.1 Modesof Animal Communication Mode Audition Luminescense Chemical Reflected light Tactile Electric . Directionality Low-Medium Distance Far Medium Low
Medium Relevant Uses

Alann , individuality Location Mass communication Socialdistance (Box 1973 ) Contact Aggression

High Low
Medium High Low

Low Low

it to robots that are experts at assembling structures than to try to make all of them competent for both tasks. The cliche "jack of all trades and master of none" may apply to robot societies as well . 9. 3. 3 Communication

has Communication two major aspects : . Information Content Most animal communicationmechanisms : operateat . a very low bandwidth Even when vision is used the signalsbetweenagents , . in generallyhave a low information content Messages animal societiesare oftenvery limited: For ants therearetypically ten to twenty differentchemical , Is ); signals( HolldoblerandWilson 1990 andmanuna , birds, andfish havebeen estimatedto have approximatelyfifteen to thirty-five distinct major display behaviors Moynihan1970 . Note that thesemay be gradedby intensity . ( ) . Mode: Different animalsocieties a surprisinglywide rangeof communication use mechanismsincluding chemical biol~ escencereflectedlight, tactile , , , , acoustic echolocationinfrared andelectriccommunication table 9.1). , , , (
9. 3.4 Spatial Distribution

Spatialdistributionis particularly importantfor activitiessuchas foragingfor food. Spatialconsiderations include small versuslargegroupsor overlapping versusnon-overlappingforaging ranges Resource . , density an environmental

367

Social Behavior

factor, often has a direct bearingon overall society size (the more resources , the larger the group and also influencesthe foraging patterns so that the , ) more restrictedthe resource the greaterthe overlapof foraging ranges(Alt , mann 1974 Carr and MacDonald 1986 . A generalization ; ) resultingfrom this is Horn' s Principle of Group Foraging which statesthat if a resourceis , relationship , , evenly distributed it is better for the agents(in this case birds) to form individual, non-overlappingforaging rangesinsteadof roosting and ) foraging together( Wilson1975 . Variousmodelsfor ant foraging have also to beendeveloped relating foragingrangesand strategies resource densityand distribution ( Deneubourg Goss 1989 Gosset ale 1990 . Similar considerations and ; ) ' of resourcetask allocation should also affect robot societies social behavior .

9.3.5 Congregation
. Coordinatingactivity is important for a society How can the societyremain togetherover time? Simple taskssuchas finding other agentscan be difficult in a large group or broad area Animals use various strategies accomplish . to this task: . By defining a colony location as a predefinedmeetingpoint recognized by otheragentsagents converge this location. Colonieshavethe advantages can at , . of havingcommonstorage resources gooddefense of and capabilities . Lekking is a group behaviorthat involvesthe generation a loud noiseby of increasing a numberof similar agents(e.g., animalsof the samesex simultaneously ), ' the likelihood of other agents hearingandthenjoining and strengthening the group' s lekking. . Distinctive calls can be usedto help find lost agentsor to indicate that an agentis lost. . Specificassembly calls by a single agentcan also mustera group of agents . that is widely dispersed

9.3 Performance .6
To effectively evaluate societal system perfonnance , specific metrics must be introduced . One useful metric is speedup ( 5 [ i , j ]), a measure of the performance of a team of N robots relative to N times the performance of a single robot . Formally , the speedup for a team of i robots carrying out j task actions is

368

9 Chapter

P Sj]=[l,j]i, [i, P [~

(9.1 )

where P [ i , j ] is a performance measure. P [ i , j ] can be measured in many different ways, depending on what is important to the task. It could be the total time taken to complete a task the total length of travel for the robots, , the energy used during task achievement, or various combinations of these or other metrics ( Balch and Arkin 1994) . Speedup results can be categorized into sublinear performance ( 5 [ i , j ] < 1), where multiples of a single agent perform better than a team; superlinear performance ( 5 [ i , j ] > 1), where a team performs better ; and linear performance ( 5 [ i , j ] = 1), a break- even point where the overall performance is comparable (Mataric 1992c) .

9.4 WHATMAKES ROBOTIC A TEAM ?

The issueof what makesa robotic team influenceshow designers robotic of and societiesmakeintelligent decisionsregardingwhat organization , systems communicationbehavioralstrategiesandthe like areappropriate aparticular for , , task environmentWe now examinesomewaysin which robot teamscan . be structured in so doing help definethe designspace thesesocieties and for . in , ; Early researchers Tsukuba Japan(Premvutiand Yuta 1990 Yuta 1993 ), several the organization multirobot of categorized importantaspects regarding teams delineatingeachsocietyaccordingto , . Active or non-active cooperation Robots either shareor do not sharea : commongoal. . Level of independenceControl is either distributed or centralized (the : ' decisionsare madeeither robots ) locally or by someexternal global agent or somecombinationof both. . Typesof communicationCommunication either : is . explicit (wherea signalis intentionally shared betweentwo or more robots ) or ' . implicit (where information is sharedby observationof other agents actions ). . Thesefirst stepstowardsa taxonomyaswell asa proliferationof multiagent roboticsresearch a Canadian led research et ) group( Dudek al. 1993 to propose a more completetaxonomycapableof categorizingthe variety of multiagent robotic systems aroundtheworld. It characterizes beingcreated laboratories by theselines: teamingalong

369

Social Behavior

. Team size: Refers to the number of robots and consists of the following es: subclass alone (one robot ), pair (two robots), limited group (a relatively small number of robots given the magnitude of the task), or infinite group (for

all practicalpurposes infinite numberof robots . an ) . Communication range: Refersto eachrobot' s ability to communicate directly with otherteammembers consists the following subclass none and of es: (no direct communication, near (only robots within a short distancecan be ) ' communicated with directly), and infinite (no limit to the robots direct communication ) capabilities. . Communication topology: Refersto the pathwaysby which communication can occur The subclass are broadcast all information is sent and received . es ( is ), ( by all robotswithin range addresseddirect messaging allowedon a namedbasis tree (only hierarchicalcommunication permitted and graph is ), ), communication canbe established . (arbitrary ) pathways . Communication bandwidth: Refersto the amountof communication available . The subclass are high (communicationis for all practical purposes es costsare approximatelythe free), motion-related(motion andcommunication costsarevery high), and zero(no communication same low (communication ), is available. ) . Teamreconfigurability : Refersto the flexibility regarding structureand the es are , organizationof the team with subclass static (no changes permitted), communicationcoordinated(robots in communicationwith each other can is ) reorganize, anddynamic(arbitraryreorganization permitted). . Team unit processing ability : Refers to the underlying computational model used Subclass include non-linear summationunits, finite state automata . es . , , pushdown automataandTuring machineequivalent . Teamcomposition: Refersto the compositionof the agents themselvesThe . morethanone arehomogeneous the same andheterogeneous all subclass es ), ( ( type). Cao et al. ( 1995 at UCLA' s CommotionLab madeanotherattemptat capturing ) four principal research . the designspaceof multirobot systems It describes axes : ' . Architecture Whetherthe systems control is centralized decentralized . or : ' structureandcontrol . Differentiationof agentsWhetherthe constituent : agents are ). ) or systems identical(homogeneous different (heterogeneous . Communication structures : . Via environment for example a trail left by the robot : , ' . Via sensing by observingotherrobots actions :

370

Chapter9 . Via communication : by intentional signaling . Models of other agents' intentions , capabilities , states, or beliefs : This aspect incorporates ideas from the distributed AI community . It is probably premature to assume that anyone of these categorizations or taxonomies can adequately express the wide range of robotic team possibilities . In the remainder of this chapter, we will focus instead on social organization and structure, interrobot communication , distributed perception , and societal learning , concluding with a case study of a successful application . As in previous chapters, despite the large body of simulation studies, we concentrate on those systems tested on actual robotic hardware.

9.5 SOCIAL AND ORGANIZATION STRUCTURE

' The behavioralarchitecturefor a robotic societys constituentagentsis only oneof manycommitments madeduring teamdesign The permissible . communication betweenteammembers the societalstructure( homogeneous and protocols or heterogeneous . ) agents arealsoextremelyimportant We now look at a for . discussed rangeof architectural strategies robotic societiesOftenthe issues ' within theseparticularsystems transcend individual agents behavioralarchitecture the . The systemsdescribedare only representative the field and do of not constitutea completesurveyby any means .

9.5 The NerdHerd .1

-style An intellectualdescendent Brooks Matarichasexpanded of , subsumption architectures applicationsof robotic teams( Mataric1994a. We encountered for ) -basedforaging this multiagentapproach the contextof a subsumption in has systemin section4.3.4. A broadrangeof basicsocialbehaviors beenspec ified, including . Homing: Eachagentstrivesto moveto a commonhomebase . . Aggregation Agentstry to gatherwhile maintaininga specifiedseparation : . . Dispersion Agentscovera largearea establishing maintaininga minimum : and , between robots . separation . Following: Robotsfollow oneafter the other . . Safewandering Robotsmovearoundwhile avoidingcollisions with obstacles : and eachother . As is standard subsumptiona rule-basedencodingis used For example for . : ,

371

SocialBehavior

Figure9.3 TheNerdHerd ( photograph . of .) courtesy M. Mataric Aggregate: If an agent is outside the aggre, ation distance g turn toward the aggregation centroid and go . Else stop . Similar simplerules areconstructed the otherbehaviors for . 1\ vo differentcoordinationmechanisms used direct combination which are : , is a vector summationprocess and temporalcombination which sequences , , . information is encodedas a through a seriesof behavioralstates Perceptual - k? seriesof predicates e.g., at-home havepu,? crowded behind kin? sense ? ? ( ?) used to encodethe sensorydata neededto activatethe relevantbehaviors puck . The systemhas been evaluatedboth in simulation and on a set of up to , twenty small mobile robots the so-called Nerd Herd (figure 9.3) . The basic behaviors describedearlier can be combinedto yield more complex social , , interactionsincluding , . flocking, consistingof safewandering aggregationand dispersion . , ,

372

Chapter9

. surroundingconsistingof safewandering following, and aggregation . , , . herding consistingof safewandering surroundingand flocking. , , , . foraging, consistingof safewandering dispersion following, horning and , , , . flocking Note that in herding for example compositebehaviorssuch as surrounding , , are alsousedasbuilding blocks. The contributionsof this work lie not in architecturaladvances rather but in the study of new rule-basedbehaviorsfor multiple physically embodied . agentscapableof interactingwith eachother This social interactioncan lead ' with eachothers goals complicatingoverall societal to physicalinterference , taskcompletion .
9.5.2 Alliance Architecture

Another offshoot of the subsumptionapproachis the Alliance architecture , which includesspecialconsideration heterogeneous for teams robots(Parker of 1994 1995 . Alliance varies significantly from subsumptionin its addition , ) . of behavior sets and a motivational system Behavior sets enabledifferent , groupsof low-level behaviorsto be activetogetheror to hibernate permitting -style architecturesMotivational a reconfigurability atypical of subsumption . behaviorsenableor disable thesebehaviorsets They operateby accepting . , in additionto the normalinputsfrom sensors inhibition from otherbehaviors and in architecturesinformation from interrobot , commonplace subsumption ' . communicationand the existing agents internal motivational state Internal motivation allows the robot to respondeffectively when trappedby permit(give up) on a task ting it to becomeimpatientor by allowing it to acquiesce if it is overly difficult or unachievableTo someextentAlliance canbe viewed . that asaddinga layer abovethe subsumption architecture embodies thesenew 9.4). capabilities(figure ' The direct input of communication signalsfrom otherrobotsinto an agents . active behaviorsfacilitates cooperationbetweenagents Explicit models of interrobotcommunication supportinginformation transfer provide predicates two robotsregardinga specifictaskover a given time period. between ' Impatiencerelates to a robot s waiting for the completion of a task by anotherrobot that is a prerequisitefor the impatient robot' s next action. In Alliance, it is implementedusing an impatiencerate function and a binary _ impatienceresetfunction. Thesefunctions control the motivationalvariable is representing impatiencewithin the robot. Acquiescence similar but deter

373

Social Behavior

Figure 9.4 The Alliance architecture .

to mines when to changebehaviorin deference anotherrobot. Each robot' s (ri ) overall motivation for a behavioralset Ail is computedby the following : equations ) mij (O = 0, and mij (t ) = [mij (t 1) + impatien (t )] * sensory Ceil Jeedbackij (t ) * _ ( activity_suppressionijt ) * impatienceresetij(t ) * acquiescen (t ), Ceil where impatien (t ) is the impatiencerate function that determineshow Ceil ; impatient sensory quickly the robot becomes Jeedbackij (t ) is a binary predicate for that indicateswhetherthe preconditions the behavioralsetaresatisfied or not; activity_suppressionijt ) is a binary predicate ( indicatingwhetheror not anotherbehavioralsetaik, j ~ k is activeat time t ; impatience resetij(t ) is a _ on robot is makingprogress thetaskthat that is 0 whenanother binarypredicate therobot is waiting on, andotherwise1; andacquiescen (t ) is a binarypredicate Ceil that detennines whetherto give up on a task or not. Thus the motivation dataindicatesthat it is for a behavioralsetwill continueto grow unlesssensor

SENSORS

C
3 Set Behavior

Set

Behavior

COMMUNICATION
-

'

"

Inhibition , -

Lateral " , , ' , ,

374

Chapter9

Table9.2 tasks Alliance for Example

Task Box pushing Robots Behavioralsets

, -home Genghis pushgo -left, pushright R-2 push -2 -locations -methodically Hazardous wastecleanup 3R find ,
Janitorialservice Boundingoverwatch

-wander find locations , movespill, reportprogress -garbage -furniturecleanfloor Simulation empty , dust , -leaderfollow leader Simulation join-groupemerge , , , leadto-waypointoverwatch ,

not neededanothercompetingbehavioris active anotherrobot hastakenover , , the task or the robot givesup on the task. When the motivationvaluecrosses , an arbitrarily predefinedthreshold then behavioralset Ail becomes active in , robot ri . The robot then concurrentlyand periodically broadcasts all other to robotsthe fact that Ail is active . Alliance hasbeenusedfor a wide rangeof mission scenariosas table 9.2 , shows Figure9.5 showstwo snapshots thehazardous . of wastecleanupmission . beingconducted threeR-2 robots by

9.5 Stagnation .3 Behaviors

Kube and Zhang ( 1994 of the University of Alberta havestudieda common ) : . problemin multirobot tasks avoidingstagnation during taskcompletion Stagnation occurswhen team membersare not cooperatingeffectively with each other Alliance address this problemthroughmotivationalvariablessuchas . es and , acquiescence impatience relying to a large degreeon interrobot broadcast communication An alternativeapproach . addsa new stagnation behavior to the overall architectureconsistingof one or more specific strategies used to overcome particulardifficulty confrontingthe team In box pushing for the . , ' , example stagnationmay result from individual agents pushing in opposite . directions effectivelycancelingeachother' s forces , for 9.6 depictsa three behaviorarbitration basedarchitecture a boxFigure task. Stagnation definedin this contextaswhena robot is in contact is pushing with the box, but the box is not moving. To handlethis potentialevent each , ' behavioris composed severalstrategicbehaviorsincluding of agents stagnation the , , realignmentwhich changes directionin which therobot is pushing and

375

Social Behavior

(A)

( B) Figure 90S In a mock-up mission a team of robots (A ) retrieveand ( B) deliver spill objects(the , dark pucks to the disposalarea(the squareregion in the foreground. ( Photographs ) ) . courtesyof Lynne Parker)

376

Chapter

Figure 9.6
Behavioral architecture . incorporating stagnation behavior .

, repositioning which movesthe robot to a different randomlocation alongthe . box' s perimeter Thesestagnation can strategies be assigned priorities according to the length of time the stagnation condition haspersisted (In this case . , is attempted beforerepositioning) . realignment In contrast to Alliance, no explicit communicationis required between ' robots nor knowledgeof the other agents intentions to eliminate the stag , ,
nation condition . The behavioral control architecture has been success fully verified in experiments with teams of small robots (figure 9.7) . 9.5.4 Societal Agents

-basedrobotic architectures havealso beendeveloped and Multiagent schema fielded. In the SocietalAgent Theory ( MacKenzie1996 a single representational ), behaviors syntax is usedto expressnot only primitive sensorimotor andassemblages alsoteamsof physicalagentsThis approachinspiredby but . , ' ), Minsky s Societyof Minds ( Minsky1986 makesno distinction betweenthe behaviorsA societyconsists a . of methods usedto deployintraand interagent that collectionof behavingagents mayor may not be spatiallydistributed(i.e., ). mayor may not havemultiple physicalembodiments A team of robots can itself. thusbe viewedasan assemblage

" ACTUATORS

'

"

. .

rAI

ON

"

. . .

J . 1 . ,

. _ .

. " -

. , -

_ -

. " " ,

" " , -

" _ -

~ I . 1 . I

STAGNATION r I I I I I I 8 8

I I I

377

SocialBehavior

Figure 9.7 A teamof smaIlrobotspushinga box.

Earlier work at GeorgiaTech(Arkin 1992b providedconceptual ) proof that robotic teamcooperation was feasiblein the absence any explicit robot-toof robot communicationThis wasshownfirst for foragingtasks thenextended . to , ( ) consumingand grazing scenarios Balch and Arkin 1994 . A straightforward -basedreactivecontrol (section4.4) was used but there extensionof schema , wasno direct modelingof the robot societyasan entity itself. The subsequent SocietalAgentTheoryprovidesa new means expressing for both homogeneous heterogeneous and teams This in turn facilitatesthe design . of robot teamsand hasbeenincorporated into a multiagentdesignand specification Lab (section9.9.2) ( MacKenzieCameron and , , systemcalledMission Arkin 1995 . ) Multiagent behaviorssuchas formation control, which allows cooperative motion of robots relative to each other ( Balch and Arkin 1995 and team ), , teleautonomy where a single human operatorcan effectively influence the

378

9 Chapter

(A ) Robotsin line formation. Figure 9.8 1 /0 DenningRobotsmoving acrossthe laboratory initially startingin line fonnation \1 , (side by side then transitioningto column formation (one following the other - note ), ) in change orientationrelativeto stripeson floor.

behaviorof an entire team of robots (Arkin and All 1994 havebeendeveloped ), within this framework (sections9.9.1 through 9.9.3). Thesebehaviors havebeentestedin simulation on Denningmobile robots(figure 9.8), andon , ' military vehiclesas part of the DefenseAdvancedResearch ProjectAgency s ' s Unmanned GroundVehicle( UGV DemoII Program(section9.9). (DARPA ) )

9.sis Anny Ant

Project

At Virginia Tech Johnsonand Bay ( 1995 have focusedon cooperationby , ) teamsof robotsin payloadtransportationA controller (figure 9.9) consisting . of four behaviors coordinationmechanism been has using a vectorsummation for developed eachof the agentsto direct a transporttask that involveslifting a pallet containingmaterial and moving it to a goal location. The orientation behavior strives to keep the pallet level, independent its height. A force of

379

SocialBehavior

( B) Robotsin columnfonnation. ) Figure 9.8 (continued

behavior coordinatesthe forces exertedby the individual robot with those of other membersof its team using interagentbroadcastcommunicationto . distributethe load as evenly as possible The pallet-contactbehaviorensures that the robot maintainscontactwith the pallet as it moves while the height , behavior detenninesthe level at which the pallet should be held. Although testedonly in simulationto date this research , providescompellingresultsfor in lifting , transporting andlowering payloadsover rough terrain. , cooperation

9.6 INTERROBOT COMMUNICATION

in between robotsis an extremelyimportantconsideration the Communication of a multirobot society In this sectionwe will considerthe following . design issues : . . . . at Whethercommunication needed aU is shouldbe pennitted Over what rangecommunication What the informationcontentshouldbe and What guarantees be maderegardingcommunication performance can

'

"

"'

' ;

"

"

. '

. ' r &

380

Chapter9

S E N S 0 R S
Figure 9.9 Behavioralcontroller for payloadtransport . 9.6.1 The Need for Communication

1:

ACTUATORS

A fundamentalissue in effecting cooperativebehavior in a team of robots is the level of appropriateinteragentcommunication Communicationis not . free andcanbe undependable can occurexplicitly, throughdirect channels . It , or indirectly, through the observationof behavioraldisplaysor changesleft in the environment(e.g., trail marking . In hostile environments electronic ) , countermeasures be in effect, lamming information flow betweenagents may or introducingdeceitinto the informationstream . So what should a robot listen to and believeif it is to work togetherwith otherrobots How shoulda designer a multiagentrobot systemincorporate ? of communication the system Identifying the major rolesof communication into ? in robot teamsshouldhelp in the designprocess They include ( Fukudaand . ) Sekiyama1994 : . Thesynchronization action: Certaintasksrequirecertainactionsto beperformed of in a particular sequence simultaneouslyCommunicationbetween or . theseactivities . agents providesthe ability to coordinate . Information exchange Different agentshave varying perspectives the : on world basedon their spatialposition or knowledgeof past events It is often . usefulto sharethis information . . Negotiations Decisionsmay need to be made regardingwho should do : what. This avoids the duplication of effort yielding a more efficient society . The communication sharingof goalsandintentionscanleadto productive and ' in . changes behaviorbasedon other agents projectedactions Is communication ? ) importantfor cooperation WernerandDyer ( 1990 have studied the evolution of communicationin synthetic agents They demonstrated . that directional mating signalscan evolve in thesesystemsgiven the

381

Behavior Social

. of ( ) presence societalnecessityMacLennan 1991 hasalsostudiedthis problem can that andhassimilarly concluded communication evolvein a societyof simple in evolved In his studies the societies which communication roboticagents . , . were 84 percentfitter than those in which communicationwas suppressed Institute of conductedat the EnvironmentalResearch In simulation research multiagent ) Michigan, Franklin and Hannon ( 1987 useda rule-basedcooperative to study the role of communication cooperation and inference , , system of lead to specialized and how theserelationships categories cooperative systems . Regardingcommunication they recognizedthat information neednot , be explicitly requested a receiverto be potentially useful to a multiagent by systemas a whole. All of thesestudiesarguefor the utility of somelevel of . in communication robotic teams NonethelessArkin ( 1992b hasestablished , ) of , , that for certainclasses tasks explicit communicationis not a prerequisite . for cooperation

9.6.2 Communication Range

is A tacit assumption often madethat louder is better that is, that the wider a ; will be. This is not robot' s communicationrange the better its performance , 1995a studied calls for help in the the . ) necessarily case Agah and Bekey ( contextof a multirobot object carryingtask at the Universityof SouthernCal ifornia. In their tropism systemcognitive architecture a set of attractiveand , ' aversiveactionsis selectedbasedon an agents current sensoryinput. From this set, a single action is chosenbasedon a weightedroulette wheel strategy ' . into a that incorporates degreeof nondetenninism the robot s responseIn to one example two small robotshavebeenconstructed carry out the task of , foragingtaskusinghomogeneous pipe transport(figure 9.10). In a simulatedcooperative can that robots it wasdemonstrated societalperformance decrease , ' s communication . radius The trade off is in with increases a robot substantially an that too weaka call for helpprevents agentfrom beingheard but too strong , effectiveexplorationof the a call bringstheentirecolonytogetherandprevents . . environmentLoudestis indeednot bestfor all tasks the optimal communication to A probabilistic approach determining range for multirobot teams under different conditions appearsin Yoshida et al. . 1995 This rangeis detenninedby minimizing the communication delay time . betweenrobots assumingthey are moving randomly If more robots send , information than the receivingagentcan handle resultingin completeblockage , as of communicationflow, the optimal range Xoptimal (represented the

382

Chapter9

Figure 9.10 lWo small robotscooperatively carrying a pipe. ( photograph courtesyof A. Agah and G. Bekey) .

numberof robotswithin the outputrange is computedasfollows: average ) ! = .vI C (9.2) Xoptimal - , P where c is the information acquisitioncapacity an integer representing the , limit on thenumberof robotsthatcanbereceived anyonetime without at upper loss of information, and p is the probability of information output for each robot.

9.6.3 Communication Content

But what should be said betweenrobots Yanco and Stein ( 1993 at MIT , ? ), studiedcommunicationspecificallyin the contextof robotic systemsIn their . research a task is defined that requires communicationto coordinatetwo , robots Ernie (the follower) and Bert (the leader (figure 9.11 . The robots , ) ) have an extremely limited vocabulary(two words that self-organizesover ) time, improving the performanceThe targettaskinvolvesthe follower robot' s . ' the leaders behaviorby either spinningor moving forward. Both mimicking

383

SocialBehavior

9.11 Figure
Ernie andBert. ( photograph . courtesyof Holly Yanco)

TASK SIGNAL REINFORCEMENt SIGNAL

LEADER ROBOT ROBOT LANGUAGE SIGNAL FOLLOWER ROBOT

ACTION

ACTION

Figure 9.12 . Learningcoordinationvia communication

robotsreceivereinforcement from a humaninstructor Figure 9.12 depictsthe . betweenthe robotsandinstructorprovidedreinforcement . relationships 's At GeorgiaTech researchers studiedcommunication impact on the performance , of multiagentrobotic teams Initial studies(Arkin 1992b indicated . . ) in of that robots could cooperate foraging tasksevenin the absence explicit communicationCooperation this contextis evidenced the phenomena . in as of effort of manyrobotsto perform a task. Holldobler and recruitment the shared , that Wilson ( 199Op. 265) havedefinedrecruitmentas" communication brings , in spacewherework is required" Although communication nestmates somepoint to . the at mechanisms enhance speed which multiple agents can converge of at a commonwork location recruitmentlike behaviorin the absence direct ,

384

9 Chapter
communication between agents alsobeendemonstrated the has . This resultargues that althoughcommunication be useful it is not necessary certain for , may . typesof tasks For example in a foragingtask, eachof the agentsoperatingindependently , , , can discovera commonattractor As discoveryoccurs more agentsacquire . , the sameobject and work togetherto transportit to a commongoal. As they on , converge the attractingobject the speedat which it is retrievedincreases because the larger numberof actorstransportingit, yielding a cooperative of effect. In mostcasesobjectstoo largefor movement a singleagentcanstill , by be recovered success havearrivedat the work site. fully after multiple agents But what is gainedif communication ? ability is added Embarkingfrom the minimalist approach(i.e., what can be accomplished the absence any in of communication, additional studieswere performedto quantify performance ) basedon addingexplicit communication foraging, consuming to improvements and grazing tasks(section9.1). 1\ vo new classes communicationbetween of , wereintroduced(Balch andArkin 1994 : ) agents . Statecommunication A single bit of information is transmitted indicating : , which states) the transmittingagentis in. Figure 9.13 showsa partitioning of ( the foragingFSA, wheretransmission a 0 indicatesthat the robot is wandering of of : , and transmission a 1 indicatesthat it is goal-directed either acquiring the detectedattractoror returning it to the homebase Insteadof headingdirectly . toward the attractor object, the robot movestoward the transmitting , agent following it until within detectionrangeof the object itself. This type of communication analogous displaybehaviorin animals is to . . Goal communication Going one step further, the location of the detected : attractoris transmitted the attendingagent Herethe robot canmovedirectly to . to the goal objectwithout following the otheragent . Someexamples from simulationstudiesqualitativelyshowthe variationin with thesemethods figure 9.14 for foraging behaviorshowsin : performance ' -like a Rorschach displaythe reductionof effort in accomplishing societys the taskasmorecommunication introduced figure9.15, showingthe consuming is ; task, depictsa markedreductionfor statecommunicationbut no noticeable differencewhen statecommunication replacedby goal. Theseresultswere is to Denningmobile robotsfor further experimentation (figures9.16 and ported 9.17 . ) Quantitativeanalysisof extensivesimulationstudiesyielded the following conclusions content( BalchandArkin 1994 : ) regardingcommunication

385

SncialBehavior

Figure9.13 communication task State duringforaging .

. Communication improvesperformance significantlyin tasksinvolving little communication ) (foragingandconsuming. In the grazingtask, robots implicit . leaveevidenceof their passagesincethe placesthey visit are modified This , . fact is observableby the other robots These types of communicationare act . referredto asimplicit, sincethey requireno deliberate of transmission . Communicationis not essentialin tasksthat include implicit communication . . More complexcommunication (goal) offer little benefitoverbasic strategies , (state communicationfor these tasks confirming that display behavior is ) . method indeeda rich communication
~ ( z 0 a I

a w ~

9.6.4

Guaranteeing Communication

Fonnal theoreticalmethodshavebeenappliedin a limited way to ensurethe . quality of communicationin multiagentrobotic systems At the University of California at Riverside Wang( 1995 has looked at distributedmutual exclusion , ) ' . for , techniques coordinatingmultirobot systems In Wangs research . no centralizedclock or sharedmemoryis usedbetweenagents Only limited is communicationin immediateneighborhoods requiredto provide deadlock

386

Chapter9 State Communication

, . 1 { . . , ' .

No Communication

(A)
Goal Communication

(B)

Figure 9.14 . , Typical run for foraging task with (A ) no, (B) state and (C) goal communicationThe showthe pathstwo robotstook in retrieving sevenattractors Note that moving . figures from (A ) through(C) the societybecomes more goal directed The simulations . progressively . , , , required 5, 145 4,470 and 3,495 steps respectively to complete Additional communication . consistentlyimprovesperformance

'

" . '

' , 1 . ; ' " " ~ " ~, ,

"

: ' "

" , , . , " " : , , I ; / ~ . , I ' . . ; . . . I . ' ' , , Y ' " ~ ' . " ' , ' ' a . , ' . ' ' . ) . C ( . ' "

. . . ~ . . " I . :

: -

. '

. " 1 '

w "

: . . , ' " ' , J

~ . : I .

~ -

, C , , ' , ' ' . , . \ v , , . . . , \ \ , ' ' , " . r . " . " ~ , . ' ' " I q I . . ' , ' OJ , , ' . / ~ ' _ r " ~ ! J , , , ~ ~ ~ ' . . ~ " , , . / / / . r , , . " ~ . , ~ ~ , W . \ . . , I ' ~ . I , . ; , ; , , , \ . " , , . ' , ' , I ~ ] ; I ~ ' 101 ! i " / \ I 1 , . . ' , ' , . , , ' . , . . ' \ ' / I \ ' : ' , " . . ' ~ \ I ' " " , , ~ . ' ' ' " ' A " ' " . ' ' ' , ' " . , " ' " . " r . , " " ' ~ ' r . " . , , , . . I - I ' - - ' ~ ! , . . . , " ~ , , : ' : . . , ' , ' . . ' , ' . r ' ' -

" '

'

. .

. . . ,

"

' , r J " I . " ' . '

. ' " ,

" ' ' , " " . . ~ . . . , . ". " ' - ' > ' ' ' " . . " . ~

"

: ' ' " ~ , ' ,

'

\ ' '

"

\ . . .

/ " ~ ' \ , ' " , " ' " , , , j . : , I

'

'

" " " . . ~ ~ ' " ~ ' _ . , . \ , '

\ ' . . ' , " ) " ' . " " . " " , . . ' " ~ . ' " ~ - . , , , I " " \ \ 11 ~ . ' " . . , , . . " ' ' r ~ ' I . , , , . , r Ij ( , ~ . , . " ' " , ~ . ; . , " , ' : \ : . ~ ' " r , .~ \ . . , . . ~ \ ~ . ' f : . * ~ . ~ A " ' . ' ' ~ . ' ~ . . ~ ~ _ . . . , I ' . , / " , , " ' jJ , " ' ;~ ~ t . ; ' , . . L . " : . ' , . ~ . . ~ ' C , ~ / ~* . ~ ~ ' - i ' J . , . . , , ,

"

, '

\ , , .

, . I

Goal Communication

~ . , . \ \

, ~ ,

\ ~

I . . . . . . L ' , I \ . 4 , ' ; , I " ; . . . ' . , < c ' ; . -

' Y

. ~ . " , " , ~'

" , , , r ' , . I " " I " ' ~ . , , _ " ' . . . . / 8 I , " , , " " , , , . \

(A)

No Communication

387

Figure 9.15 The consumingtask with (A ) no, ( B) state and (C) goal communicationThe simulations . , . , , required9,200, 8,340 and 8,355 steps respectivelyto complete Note that state andgoal communication are performance approximately equalin this task.
. . . . . .

SocialBehavior

State Communication

(B)

, . II , ' . ~ I I 1 r " , , : " \ 1

388

Chapter9

(A)

( B) Figure 9.16 the , , , (A ) Two Denningrobots Ren and Stirnpy demonstrate foraging task in this case . . without explicit communication( B) Rentagsan attractor

389

SocialBehavior

) (C

(D) ) Figure9.16(continued " " .( ) deliver attractors home . the to base (C) Stimpytags anattractor D ReoandStimpy

391

SocialBehavior

(C) ) Figure 9.17 (continued A reconstruction the pathtaken(from previousfigure) in the foragingdemonstration of . Note the cooperation retrievingthe last object in figure (C). in

detection(a stagnationcondition or to coordinatemultiple agentscompeting ) for a single resource(e.g., passingthrough a narrow corridor or crossing " at an intersection. Robotsuse" signboard communication Wang 1994 ) ( ), -bandwidth message a specific low protocol displayedby a device on each robot and perceivableonly by nearbyrobots Although the results have yet . to be fielded on actualrobots they can be provencorrectwithin their formal , framework . Lin andHsu ( 1995 at the NationalTaiwanUniversitydeveloped adeadlock ) free cooperationprotocol for a multiagentobject sorting task. The protocol usesbroadcast communication sort agentpriorities after a deadlockcondition to hasbeendetectedSpecialized . behavioralstrategies havebeendeveloped -balancing among the for helping other robotic agents for performing load , robots and for selectingpartnersfor the task of moving a set of randomly , . , placedobjectsto specific goal locations An object 0 ;, requiresi robots to moveit to its goal location. Deadlockcaneasily occur whenthe robotsdo not . help eachother This is a variationon the foragingtaskandcanbe represented asan FSA, asin figure 9.18.

Base -

Horne V ' / ' " _ ~ . . . . . . . . . . ~ ~ ~ " Obstacle h ~ L r actor ' D Atb

392

Chapter9

START -arid receive -help accept request -home reach

objec -help enough

9.18 Fil Ure . Simplified FSA for object sorting A help requestis emittedwhen the robot entersthe wait stateafter detectingan objecttoo largeto moveby itself.

for The robots are capableof broadcasting requests help and sendingout -to-point offers to help in response particular requests In one strategy to . point , deadlock is preventedby establishingpriorities for the objects to be movedand helping to move the one with the highestpriority first. Priorities ' . are basedon the object s distanceto its final destination If there is still a -coordinatesare compared and if still needed y-coordinatesare , , conflict, x the used for tie breaking Since no two objects can possess samex- and y. are coordinatesdeadlockcannotoccuraslong assufficientagents availableto , involve detecting movethe largestpossibleobject. More complexstrategies help deadlockconditionsafter they occur and then remedyingthe situationor of generatinga feasiblesequence actionsthat preventsdeadlockfrom occur balancingthe workload amongthe available ring at all while simultaneously robots .

PERCEPTION 9.7 DISTRIBUTED

have focusedon how information can be of Our discussions communication . sharedamongrobotic agents Further issuesinvolve how perceptualactivity

393

SocialBehavior

can be coordinatedamonga team of robots what sensoryor perceptualinformation , is worth sharing and how a team of robots as distinct from an , , individual robot, shouldview the world. One important perceptualtask relating to the notion of perceptualclasses , section7.4.3), is that of distinguishingteam membersfrom other environmental ( . features Kin recognitionis the term usedto refer to this particularperceptual neuralcircuitry often ) ability (Mataric 1993b. In biology, specialized es , the , accomplish this task. Prosopagnosia inability to recognizefaces occurs in humans whentherearelesions typically dueto strokes on the underside of , , theoccipitallobes providing strongevidence " someneuralnetworkwithin that , " this regionis specialized the rapid andreliablerecognitionof humanfaces for , (Geschwind1979 p. 112 . ) In multiagent robotic systems kin recognition mayor may not be useful , . on the circumstancesA robotic team that does not have , depending often has only two distinct perceptualclasses obstacles this ability : and target . from obstacles Kin . objects Team memberrobots are indistinguishable , however when implemented provides information regarding , , recognition ' other robots position and if needed their identity. Thus behaviorscan be , , that allow the robots to interact more effectively and minimize developed interferencewith each other (Mataric 1992c. This information can be provided ) in various ways: for example by transmitting the positional information , , ) directly (Mataric 1992c or instead using specific perceptualcues such as making the robots a unique color relative to their environment(e.g., green in the caseof the 1993AAAI mobile robot competition( Balch et al. 1995 . ) The information sharedby perceptionbetweenagentscan go far beyond ' . simply recognizingeach other; an agents intentions can also be discerned information Cooperationby observationrefers to this sharing of perceived . in the absence explicit communication More specifically cooperationby of , ' observation involves observinganotheragents action then choosingappropriate , actions based on the observedaction and the current task situation ) (adaptedfrom Kuniyoshi et al. 1994 . This is closely related to the notion ' of plan recognition in the distributed AI community in which an agents , intentions are inferred by observing its actions (Huber and Durfee 1995 . ) ' In work at Japans ElectrotechnicalLaboratory Kuniyoshi ( 1995 using a , ), team of small robots equippedwith stereovision, has defined severalperceptual functions to accomplishthe task of figuring out what anotherrobot is doing:

394

Chapter9

' . Find: Directsthe observingagents attentionto a new target . . Track: Follows the targetagentasit movesthroughthe world. . Anticipate: Recognizes situationsand potentialcollisionsor otherhazardous them. prevents . Event Detection Recognizes : certain preconditionsto synchronous action ' with other robots This may involve spatialcoordinationor a teammembers . release an object. of The eventdetectionalgorithmsmust be designed a task specificmannerto in for providethe informationnecessary thejob at hand(section7.5.1). A numberof kinds of applications requiremultiagentperceptionincluding . Convoying A commontask for teamsof robotstravelingfrom one location : to anotheris follow -the-leaderor convoying This task hasusein both intelligent . vehiclehighwaysystems military logistical operationsIn somecases and . , -robot systemmay be deployed with the humandriver leading a mixed human , the convoy and the other robots following safely behind One representative . at for , developed the University of Tennessee two heterogeneous perceptual system robots usesa 10DOF robotic headperfonningcorrelationbasedvisual , tied to a fuzzy logic controller(MarapaneHolder, andTrivedi 1994 . , tracking ) Numerousother examplesinclude work at GeorgiaTech (Balch and Arkin 1995 that usesGPSsensordatafor maintainingcolumn formation in a team ) of unmanned groundvehicles(section9.9.1). . Landminedetection This task involvescoordinaed spatialexplorationand : t . probing of a geo graphic ally boundedregion. A wide range of sensorsis available for this task including magnetic Xray , acoustic subsoil sensors , , and ground penetratingradar systems A team of UCLA and U.S. Army . researchers Franklin Kahng and Lewis 1995 have proposeda prototype , , ( ) heterogeneous society consisting of ten R-3 robots and a small X -Cell 60 aerial robot. Clearly for this mission it is crucially important that agents , , shareinformation to prevent their traversingan alreadydetectedmine and also to increase efficiencyand coverage an areaby avoidingredundant the of search . . Reconnaissance surveil and lance Teamsof robotsconcerned : with monitoring an areafor incursionby anintrudermusthavethe ability to coordinate their andperceptual activities It is not advantageous robotsto spendtime . for spatial . ) looking at the samelocation while ignoring others Gage( 1992 hasdefineda methods lance , varietyof coverage prescribingthe locationof robotsfor surveil including

395

SocialBehavior

. blanketcoveragein which eachrobot takesup a stationandremainsthereto , watchfor intruders . . barriercoveragein which a staticline of robotsis created preventcrossing to , of the barrier without detection . . sweep , coveragein whIcha teamof robotsmovesthroughan areaattempting to ensurethat no enemyor intruderactivity is present . Sensorpointing must be controlled in a manner consistentwith the other ' robots positions and sensordeployments Section9.9.1 describesdecision . theoreticmethods accomplishing for this. . Map making: Expandingupon the notion of cooperationby observation , behaviorbasednavigationusing sharedmapsby a team of robots has been ) developedin a joint U.S.-Japaneffort (Barth and Ishiguro 1994 . In conin . trast to the use of panoramicvision for global localization discussed section 5.2.2.1, theserobots can cooperateby providing to one anotherinformation . regarding the relative whereaboutsof team members Aschema based control approachis used .with behaviors including avoid obstacle , , -other robot, group-momentattraction (drawing the robots toward the avoid center of mass of the other perceivedrobots to help keep the group together ), and object rangeuncertaintyattraction(guiding the robot toward regions of uncertaintyto gathermore information . Small mobile robots with ) to (figure 9.19 are being developed provide exploration ) panoramicvision systems andformationcapabilitiesthat realizethe resultsof Barth andIshiguro' s simulations .

9.8 SOCIAL LEARNING

Teamsof robotsoffer new opportunitiesfor learning particularly how to selforganize , andbecomemorecooperative over time. Mataric ( 1994b definesthe ) basicforms of sociallearningasimitation or mimicry, in which one agentacquires ' the ability to repeator mimic anothers behavior andsocialfacilitation, , in which existing behaviorsare expressed more effectively as a direct consequence of social interactions An inherenttensionexists betweenindividual . andgroup needs Agentsmay be strongly self-interested haveno concern . and ' s overall wellfor the society can being. What ecologicalpressures be brought to bearthat encourage that nongreedy strategies benefitthe societyandyet may be detrimentalto the individual and how can social rules be developedthat ' transcend individual agents goals an ?

396

Chapter9

(A) Figure9.19 with Robot vision(A) anda panoramic of a laboratoryB). view ( equipped panoramic with from IEEE) . 1994 (Reprinted pennission BarthandIshiguro . @1994
9.8.1 Reinforcement Learning

Reinforcement learning (section8.3), a commonstrategyusedfor individual robotic learning has beenapplied in multirobot contextsas well. Optimization , functions in social robotics typically center on minimizing interference ' . between and can agents maximizingthe societys reward Reinforcement result ' ' s actions of from an agent , from observation anotheragents actions , directly of anotheragentreceives vicariousreinforcement or from observation the reinforcement ( ). in Matarlc ( 1994a conducted ) experiments sociallearningwithin a foraging context A teamof four robotswasequippedwith adaptivebehaviorsfor safe . was as , , wandering dispersionrestingandhoming. Perception encoded a setof -puck? at-home nearintruder? night-time? Therobotslearned ? : predicateshave the with the appropriate overtime to associate correctperceptual preconditions . behaviorin this societalcontext Both delayedreinforcement -learning(section Q 8.3) and a progressestimatorreinforcementsummationalgorithm were

398

Chapter9

tested The progress . estimatorapproach due , yieldedbetterresults presumably to the task' s non-Markovian nature (a consequence the inherent noise in of work compared ) perceptionand actuation. Subsequent learningusing two additional social rules: yielding, in which a robot yields the right-of-way when on one side of an oncomingteammate and continueswhen on the other and , information to other robots Reinforcement . , sharing in which a robot broadcasts overthose learningusingthesesocialrulesalwaysimprovedperformance methods that usedsolely greedystrategies .

9.8.2 L -Alliance
Parker( 1994 hasextended Alliance architecture the described section9.5.2 in ) to include learning mechanismsLearning in L -Alliance involvesparametric . and , adjustment improvesteamandmissionperformanceeliminating the need for a humanoperatorto tune behavioralparameter . settings Eachteammember within L -Alliance maintainsstatisticaldataregardingits own pastperformance as well as that of eachof its teammatesThe time history is relatively . small, typically five previoustrials. This small history window permitsrapid responsiveness allows reasonable yet predictionsto be maderegardingfuture . performance requirements One learning problem unique to multirobot systemsconcernsoptimizing taskdistributionoverthe availablerobotic agentswhich is not unlike loadbalancing , in multiprocessor . , systems Indeed different task allocation strategies basedon results from the parallel processorcommunity were tested Techniques . suchastrying to accomplish longesttaskfirst (based descending the on first fit [GareyandJohnson1979 werefound to resultin terribleperformance ]) , however because multirobot taskshad a high rate of failure during execution . , Other allocation strategies shortesttask first or random selectionproduced using betterresults The learnedparametricvaluesinclude influenceandmotivation . that parameters affect task selectionandimpatienceandacquiescence valuesthat affect task completion The metricsusedto measure . performance weretime andenergyconsumption . , During the initial active learningphase the robots initially are maximally . , patientand minimally acquiescentIn the subsequent adaptivelearningphase the robots start with the parametricvalueslearnedduring the active learning . phase Ad hoc updateequationsspecific for eachof the adjustedparameters havebeenusedto achieve resultswithin 20 percentof optimal for oneparticular simulatedcontrol strategy .

399

SocialBehavior

SENSORS

9.20 Figure ( ) Tropism basedcognitivearchitecture after Agah andBekey 1997 . 9.8.3 TropismSystem Architecture Cognitive

..SYS -TRO ~ . LEAR EV P E R C MAT P TRO T I 0 N

ACTUATORS

at , Figure9.20 depictsthe ttopism systemcognitivearchitecturedeveloped the of Southern California andinttoducedin section9.6.2. A weighted University roulette wheel action selectionmechanismbasedon each action' s strength (its ttopism value arbitratesbetweenwhateverttopisms are matchedwith ) current sensoryinputs. Recall that ttopisms representthe robot' s " likes and dislikes." This architecture inttoducesthreetypesof learning(Agah andBekey 1995b: ) . In perceptual leaming, a new ttopism is created and the oldest one is , removedfrom the ttopism set The new tropism consistsof the four-tuple . , '[' , (0 , p, arandominitial where0 is the novel sensed > entity, p is the state arandom is a randomaction and '['initialis an initial ttopism value usedby the action , selectionmechanism . . Leamingfrom success resembles -learning(section8.3), graduallyincreasing Q the tropismvalue'[' by a fixed incrementup to a maximumvalue makingit , morelikely to occurthe next time the systemfinds itself in the samesituation . -tupleto a newactionwhen . Leamingfromfailure changes in arandom the four the last randomlygenerated hasprovenunfruitful. one In contrastto L -Alliance, in which eachrobot maintainsstatisticaldataon ' otherrobots performance , throughcommunicationhereeachrobot learnsindependently in of the others Performance . improvementwas demonstrated

400

Chapter9

simulation studiesusing three different metrics total number of tasks performed : of , total energyconsumption the colony andenergyconsumed task , per . completed Geneticalgori~ s (section8.5) havealso beenapplied within this architecture . Thesemethodsstronglyresemble classifiersystems their approach in , with the tropismsencodedsimilarly to productionrules. Fitnessevaluationis basedon the numberof taskscompletedand energyconsumptionThe major . 's differencebetweentheseand other GA approach lies in the performance es at being measured the societallevel, ratherthan at the individual. In addition to simulationstudies a small setof robotsin an actualimplementation learned , to gatherobjectsandcooperate carryingthem (Agah andBekey 1997. in )

9.8.4 . Learningby Imitation

' Imitation involves first observing anotheragents actions (either human or robot), then encodingthat action in someinternal representationand finally , . ), reproducingthe initial action According to Bakker and Kuniyoshi ( 1996 of observation an action involves . . . . . beingmotivatedto find a teacher a goodteacher . finding to . identifying what needs be learnedfrom the teacher ' . perceivingthe teachers actionscorrectly

of Representation an actioninvolves . selectinga suitableencodingthat matches observation the action . the to . capturinga particularobservation the chosenrepresentational in format. of Reproduction an actioninvolves . being motivatedto act in response an observation to . . selectingan actionfor the currentcontext . . adaptingthe actionto the currentenvironment . ' We havealreadyseenan exampleof action imitation in Yancos work (section 9.6.3), when Ernie the robot learnsto coordinateitself with Bert' s motor activities Another exampleof learning by imitation involvesa robot imitating . a robotic teacher ) movingwithin a maze(HayesandDemiris 1994 . During whetherthe leadereitherturns90 degrees training, the learningagentobserves . or continuesmoving forward at a given point within the maze It thenencodes a rule associating environment the with thatparticularaction. Later the agentis

401

Social Behavior

able to determine, via sensing, which action rule is appropriate for its current

withinthemaze . position

9.9 CASE STUDY : UGVDEMOn

DARPA conducteda research programin the mid- 1990sfocusedon providing . robotic systems supportfor battlefieldscoutingoperationsBehaviorbased canplaya role in this highly uncertainanddynamicdomain The UGV . clearly Demo II programemployeda team of unmanned vehiclesas scouts , ground of lance andtargetacquisitionoperations , surveil , capable conductingreconnaissance in a coordinated manner In conventional . , military operations motor ized scoutstypically move in advance the main force to report on enemy of andcapabilities . positions on Incorporated eachindividual vehicle (HMMWV - High Mobility Multipurpose WheeledVehicle seefigure 9.21) is an architecture ; consistingof a suiteof behaviors(figure 9.22). Theseinclude . Stripe A teleoperation : behaviorusedby the operatorto establishintermediate , way pointsfor the vehicle thenautomaticallycreatea paththat the vehicle strivesto follow. . Crosscountry: A path following behaviorthat usesGPSdata for localiza tion. . Ranger A navigationalbehavior using geometricdata derived from both : sensor mapdata and . . GaneshaAn obstacle : avoidance behaviorthat usesa local mapderivedfrom laserrangefinder data(Langer RosenblattandHebert 1994 . , , ) . Safety Obstacledetectionand avoidance : using stereovision (Chun et al. 1995 . ) . Alvinn: A neuralnetworkroad following behavior(section7.6.1). . Formation Formationcontrol for multiple vehicles(section9.9.1). : thesebehaviors . The DAMN arbiter (section4.5.5) is usedto coordinate At DemoA , the first of a seriesof demonstrations 1993 a singlevehicle , in , showedthe capability of road following using Alvinn and Stripe teleoperation ) (ChunandJochem1994 . Off-roadnavigational capabilitieswereaddedat DemoB in 1994usingstereovision for obstacle avoidanceChunet al. 1995 . ( ) es Sincethis chapterdiscuss multiagentsystems we focus on the capabilities , ' in for , , conducted 1995 including developed Demo C s multivehicledemonstrations and formationcontrol, missionspecification planning and , multiagent . teamteleautonomy

402

Chapter9

(A)

(B) Figure9.21 : Unmanned from the DARPAUGV DemoII program(A) Single groundvehicles . of HMMWVscout (B) EntireDemoII HMMWVUGV team(Photographs ; courtesy -Martin Denver .) Lockheed , , Colorado

403

Social Behavior

BEHAVIORS

SENSORS
SHARED

ARB DAMN ~

DS

MEMORY

. .

9.22 Figure
UGV Demo n softwarearchitecture (behavioral ). components

9.9.1 FonnationBehaviors
Fonnationcontrol hassignificantutility for a wide rangeof potentialapplications in the unmanned vehiclecommunity In UGVs, formationcontrol canbe . usedfor military scoutingmissionsandlogistical supportin convoying Scout . teamsemploy specific formations for particular tasks Column formation is . with road following activities whereas line formationsare , usually associated usedto crosslarge expanses open terrain. Robotic behaviorswere implemented of to accomplish four primary formationsfor scoutvehicleslisted by the U.S. Army manuals( 1986 : diamond wedge line, andcolumn(figure 9.23). , , ) Thesefonnation control behaviorswere success fully testedfirst in simulation : studies thenon Denningmobile robotsin the GeorgiaTechMobile Robot , Laboratoryand on two HMMWV s at Demo C of the UGV Demo II program in July 1995 in Denver Figure 9.24 depictsa sequence imagesfrom the . of demonstration which the vehiclesstartedinitially in a column formation in ,

~ 0 S ~ ~ . ~ . I ' ' ~ Ai Q ~

404

9 Chapter

.
(A)

.
(B ) 9.23 Figure

405

SocialBehavior

..

.
8

(C )

8
(D )
) Figure 9. 23(continued Four simulatedrobotsin leaderreferencedA ) diamond ( B) wedge (C) line, and , , ( turn in an obstaclefield. ( D) columnfonnationsexecutinga 9O degree

407

Social Behavior

(C)
) Figure9. 24(continued Demo Formation demo1 UGV traveling in (A ) column, ( B) wedge then C tech : \\10 s ,
(C) line fonnation.

then transitedto wedgeformation then movedto line formation and finally , , ' to . reverted columnformationwith the vehicles initial positionreversedWaywereconcurrentlyactive andthe , point navigationandformationmaintenance vehiclesswitchedamongformationssmoothlyand autonomously set GPS at ' . Differential GPSprovidedthe vehicles position relativeto designated points . eachother The formation behavior itself is comprisedof two main components a : schemadetectformation position and a motor schemamaintain perceptual ' s desiredlocation for the formation. The perceptualschema determines robot the formationtype in use its relativeposition in the overall formation andthe , , ' a otherrobots locations Maintain-fonnation computes vectortowardthis p0 . is sition whosemagnitude basedon how far out of position the robot is. Three zonesaredefined(figure 9.25) : . Ballistic zone: The robot is far from the desiredposition, so the output 's ' vectors magnitudeis set at its maximum which equatesto the schema , , gain value with its directional componentpointing toward the centerof the deadzone . computed

408

9 Chapter

. Control led zone: The robot is midway to slightly out of position, with the ' ' vectors magnitude decreasing linearly from a maximumat the zones farthest . edgeto zero at the inner edge The directional componentpoints toward the ' deadzones center . . Dead zone: The robot is within acceptable . positionaltolerance Within the ' deadzone the vector s magnitude alwayszero. is , In figure 9.25, Robot 3 attempts maintaina positionto the left of andabeam to Robot 1. Robot3 is in thecontrolled zone sothebehaviorgenerates moderate a , force towardthe desiredposition (forward andright). Each robot must computeits own position in the formation continuously . Threetechniques this computation for havebeenidentified: . Unit -center referenced A unit-centeris computed averaging x- and : the by -coordinates all robotsinvolvedin the formation. Eachrobot determines of its y own formation positionrelativeto that center . . Leader-referenced Eachrobot determines formationpositionin relation : its to a designated leadrobot. The leaderdoesnot attemptto maintainformation;

~ u " fn ~ =

409

SocialBehavior the other robots are responsible for maintaining suitable offsets relative to the

leader . . Neighbor-referenced Eachrobot maintainsits position relativeto an adjacent : robot. from this research(Balch and Arkin 1995 Someinterestingobservations ) . , provide guidancein the use of formations For example the unit-center approach requiresa transmitterand receiverfor each robot and a protocol for , exchangingposition information. On the other hand the leaderreferenced approachrequires only one transmitterfor the leader and one receiver for bandwidthrequirements eachfollowing robot, reducingcommunications significantly -restricted and making it a preferableapproachin communications . the applicationsAlso, regarding useof kin recognitioninsteadof explicit communication : unit-center referenced formationsplace a heavydemandon any e.g., vision) used In a four-robot visual formation, for . sensor ( systems passive instance eachrobot would haveto track threeotherrobotsthat may be spread , formations acrossan extremelywide field of view. Leader and neighborreferenced , , on the otherhand requiretrackingonly oneotherrobot. resources be usedeffectively can A relatedquestionis how the availablesensor . for to performefficientscoutreconnaissance a givenformation Research how to coordinateperception at the Universityof Texasat Arlington considers over multiple vehicles(Cook, Gmytrasiewicz and Holder 1996 . Figure 9.26 , ) showsa simulation of sensorpointing algorithms fielded on a team of two . robotic vehiclesaspart of DARPA' s UGV Demon program -theoretic methods utility values for individual fields-ofU sing decision , to are assigned each memberof a formation allocating specific time , regard an in eachrelativeposition (figure 9.27) . The utility valuefor scanning areaA : from position P is determined the following equation by using sensorS Uscan , S, P ) = ( E Plc (x , y) P2k(X, y ) V Ikdxdy , (A JA k (9.3)

where Plc (X, y) is the conditional probability that a targetat location (x , y ) will be identified correctly from position P using sensorS; P2k(X, y) is the prior probability that a target of type k exists at location (x , y) ; and V Ik is the value of information regardinga target of type k. Theseutility values arecontinuouslyrecomputed taking into accountdynamicfactorssuchas terrain , , security and focusof attentionandare usedto updatethe field-of-regard . selectionweights Weightedroulettewheel methodsare usedfor the selection

410

Chapter

Figure 9.26 A simulationof sensor pointing asa teamof four robotstravelsleft to right in a diamond formationconductingreconnaissance . operationsTriangleslooking outwardfrom each ' robot represent video sensorsfields of view. All targets(represented circles are the as ) success detected . fully during the mission ( Figurecourtesyof Diane Cook.) of new fields - of -regard values, introducing nondetenninism into the observational strategy that increases the likelihood of hostile target detection while . reducing the effectiveness of countermeasures 9.9.2 Multiagent Mission Specification

To effectively designa mission for a team of robots suitabletools must be , available Severaldifferent approach were developedfor use within the . es Demo II program addressing different aspects this problem This section of . focuseson the MissionLab tool set At the University of Michigan, Lee et at. .

411

SocialBehavirn

COLUMN M ON FOR An

G G Lf . ~ . :~ > ~

-"\ '~~ /"-A\ / ~, ~

UGV- 1
.

UNEFORMATION

UGV -4

UGV 2 UGV 3
Motion UGV-4

Direction

of

-~ ~"\ ", '5 /";-~ /UGV \ ~~ 2UGV 4

UGV 1
.

W "~ 7 ~ ..._7~.;.. ", ~ _ / ~ : ~

DIrection of Motion

UGV - 5 3 W ~. ) '\",-: -.7 , .~_

Direction Motion of

DIAMOND FORMATION

61 DI ction af Mauon re

--

WEDGE FORMATION Figure9.27 fieldsof regard various for fonnations number each . The in sector Example represents thepercentage timespent reconnaissance of on . activitywithinthatarea

412

Chapter9

Reasoning anothersystem UM -PRS basedon the Procedural , , ) ( 1994 developed for the current environmental plans System(section6.6.4). It elaborates . context consistentwith long-term goals Although success fully fielded on a robot namedMAVERIC (Kenny et al. 1994 only limited resultshavebeen ), . reportedto datefor its usein multivehiclemissions la at missionspecification Mission , multiagent Lab systemdeveloped Georgia -orientedphilosophyas the underlying methodology Tech usesan agent , , . of the recursiveformulationof societies robots It includesa graphical permitting , configurationeditor, a multiagentsimulationsystem andtwo different architectural the . codegeneratorsThis softwaresystemembodies SocietalAgent in Theorydescribed section9.5.4. A societyis viewedasan agentthat consists . robots Eachindividual or of a collection of either homogeneous heterogeneous in of robotic agentconsistsof assemblages behaviors coordinated various , . TemporalsequencingArkin and MacKenzie 1994 affords transitions ) ( ways as betweenvariousbehavioralstates naturallyrepresented a finite stateacceptor via can . Coordinationof parallelbehaviors be accomplished fusion (vector summation, action selection priority (e.g., subsumption or other coordination ) , ) consist . operatorsas necessaryTheseindividual behavioralassemblages motor behaviorsultimately groundedto and of groupsof primitive perceptual . and the robot' s physicalsensors actuators : a multiagentrobot configurationinvolvesthreesteps determining Creating set an appropriate of skills for eachof the vehicles translatingthosemission ; setsof suitablebehaviors(assemblagesand constructing orientedskills into ); to or selectingsuitablecoordinationmechanisms ensurethat the correct skill ' . are assemblages deployedfor the missions duration Lab is its ability to delaybinding to aparticular An importantfeatureof Mission based UGV Demo ll , subsumption behavioralarchitecture e.g., schema , ) ( . until after the desiredmissionbehaviorhasbeenspecified Binding to aparticular physicalrobot occursafter specificationas well, permitting the designto . be both architecture androbot-independent Lab system which has separatesoftware 9.28 shows the Mission , Figure . architecturesand variousrobots The librariesfor abstractbehaviors specific , , user interactsthrough a design interface tool (the configurationeditor) that . as permitshim to visualizea specification it is created asicons which canbe createdas arerepresented , graphically Specifications from an existingrepertoireavailablein the behaviorallibrary. or needed reused
via Lab I. Mission is available theworldwidewebat: - /research . Lab / Mission .html .edu .cc. ://www gatech / ai/robotlab http

413

SocialBehavior

Figure9. 28 Mission system Lab architecture .

414

Chapter9

(A) 9.29 Figure

' to , Multiple levelsof abstractionwhich canbe targeted the designers abilities, downto the low-level language rangefrom entirerobot missionconfigurations . for a particular behavior After the behavioralconfigurationis specified the , . and architecture robot typesare selectedThencompilationoccurs generating , the robot executableswhich can be run within a simulationenvironment , provided to Mission Lab or, througha softwareswitch, downloaded the actual by . robotsfor execution Panel(A ) of figure 9.29 depictsa finite statediagramspecifying a simple are . , military scoutingmission In this case explicit GPScoordinates usedas . destinationsThe remainingpanelsshow four robots during executionof the . scoutmissionin theMissionLab simulator Noticethattherobotsbeginmoving .the bottom left comer They then switch to column . in line formation from

415

Social Behavior

(B)
) Figure 9.29 (continued

formation to traverse gap in the forward lines (passage the point). The robots travel along the axis of advancein wedgeformation and finally occupy the objectivein a diamondformation.
9.9. 3 Team Teleautonomy

Another importantaspectof multiagentcontrol involvesintroducingan oper ' s intentions into an autonomousrobotic teams ' ator . ongoing performance Softwaredevelopedas part of the UGV Demo n programprovidesthis capability in two ways (Arkin andAll 1994 : )

416

Chapter9

(C )
) Figure 9.29(continued to A finite state configuration constructedwidrin MissionLab and corresponding a , of in , , scoutingmission appears (A ). The mission consistingof a sequence coordinated of actionsin differing formations is shownat variousstages executionin ( B) and (C) , , (moving initially upwardin line formation, thento the right in a column thenchanging to wedge and finally occupyingthe objectivein a diamondformation . ) ,

417

Social Behavinr

Figure 9. 30 directional control; (B) Personalityslider (A ) On-screen joystick for teleautonomous . barsfor teambehavioralmodification

. The operator as a behavior: In this approacha separate behavioris created , that permitsthe operatorto introducea headingfor the robot teamusinganonscreen the joystick (panel(A ) of figure 9.30) . This blases ongoingautonomous behavioralcontrol for all of the robots in a particular direction. Indeed all , other behaviorsare still active typically including obstacleavoidanceand , formation maintenanceThe output of this behavioris a vector representing . ' s directionalintentionsand command the operator . strength All of the robotic team membersgenerate sameteleautonomous the behavioralresponse the to ' s intentions The entire teamactsin concertwithout . operator any knowledge ' of oneanothers behavioralstate . . The operator as a supervisor: Using this method the operatoris permitted , to conductbehavioralmodificationsduring run-time. This can occur at two levels :

. ) ) B A ( (

418

Chapter9

(A)

(B ) 9.31 Figure

419
-

SocialBehavinr

0U des
~ " .

) Figure 9.31(continued Teleautonomous extricationfrom a box canyonof a teamof 2 Denningmobile robots (viewed from above. (A ) showsrobots trappedin box canyon and (B) after teleautonomous , ) removal (C) providesthe executiontraceof the robotic run (rotated90 degrees . clockwiserelativeto the photographs ).

. The knowledgeable operatorcan adjust the low-level gains and parameters of the activebehavioralset for the entire teamdirectly if desired varying the , relativestrengths behavioralcompositionasthe missionprogress and es. . For the normal operator team behavioraltraits (" personalitycharacteristics , " are abstractedand ) presentedon screenfor adjustment(panel ( 8) of 9.30). Theseinclude characteristics such as aggressiveness (inversely figure the relative strengthof goal attractionand obstacleavoidance and ) adjusting wander (inverselyvarying the strengthof noiserelative to goal attraction lust andor formation maintenance Theseabstractqualities are more naturalfor / ). an operatorunskilled in behavioralprogramming This approach . permits the concurrentbehavioralmodificationof all of the robotsin a teamaccordingto ' the operators wishes . An exampleillustrating the utility of the directional control approachinvolves . the extricationof teamsfrom potentialtraps Panels(A ) and( 8) of figure . 9.31, show(from above a run usingtwo Denningmobilerobots The active ) behaviorsinclude avoid static obstacle move to-goal, andcolumn formation . , The robotswanderinto the box canyonandbecomestucktrying to maketheir

"

"

~ I ~

T-

of . . . . . ~ t8U2 . . . . ts

420

9 Chapter . way to the goal point specified just behindthe box canyon The operatorintervenes the joystick to direct the robotsto the right. While moving, they , using continueto avoidobstacles maintainformation. Onceclearof the trap, the and to directing the robots and they proceedautonomously their operatorceases earlier prescribed goal. The overall executiontraceis depictedin panel(C) of the figure.

9.10 CHAPTER SUMMARY

. Teams robotsafford significantadvantages individual robotsin terms over of of performance sensingcapabilities and fault tolerance Problemsinvolving . , , interferencecommunication costs anduncertaintyin the actionsof otherscan , , . preventfull realizationof the benefitsof teaming . Typical generictasksfor societies robotsincludeforaging flocking, consuming of , material andgrazing . , , moving . Ethological studies provide insights into social behaviorsand interagent communication . . Multiagentrobotic systems be characterized can alongthe lines of reliability, social organization communicationcontent and mode spatial distribution , , , . , congregationandperformance . Useful taxonomies in . exist for definingthe relationships robotic teams . Behaviorbased to . architectures havebeenexpanded includesocialbehavior . The Nerd Herd andAlliance arevariantsof the subsumption architecture . . The SocietalAgent Theoryextendsschema . theory to multiagentrobotics . Communication . plays a centralrole in coordinatingteamsof robots . Communication not necessary cooperation is often desirable but . is for . Range content and guarantees communication importantfactorsin for are , , . the designof socialbehavior . Distributedperception information overmultiple robotsinvolvessharingdiscerned betweenthem. . Variousforms of machinelearninghavebeenappliedto robotic teams including , andimitation. reinforcement learning . The UGV Demo II program provides an exampleof teams of robots in of action including aspects missionspecificationformationmaintenanceand , , ,
team teleautonomy.

Chapter 10

: Behavior FringeRobotics Beyond

' " ." Theyre madeout of meat "Meat?" ' " Meat. ." Theyre madeout of meat " Meat?" " Theres no doubt about it. We ' picked up several from differentparts of theplanet, ' took them aboard our recon vessels and probed them all the way through Theyre . , " . meat completely " " That's . to impossible Whatabout the radio signals? Themessages the stars? 't come " . Theyusethe radio wavesto talk, but the signalsdon from them Thesignals come ." from machines " So who madethe machines That's who we want to contact" ? . ' ' " . . Theymadethe machines That s what I m trying to tell you Meat madethe machines ." ' " That's ridiculous. How can meatmakea machine Youre ? askingme to believein " sentientmeat . ' " I 'm not . , askingyou I m telling you Thesecreaturesare the only sentientrace in 're madeout meat" . that sectorand they of ' " , Maybe they re like the orfolei. Youknow a carbon basedintelligencethat goes " . througha meatstage " ' . . Nope Theyre born meatand theydie meat Westudiedthemfor severalof their life ' ' " , spans which didn t take long. Do you haveany idea what s the life spanof meat? 're " . . . , , Spareme Okay maybethey only part meat Youknow like the weddilei A meat . headwith an electronplasmabrain inside" " . . , Nope Wethoughtof that, since they do havemeat heads like the weddilei But I ' . . told you weprobedthem Theyre meatall the way through" , " No brain?" ' ' ' " ! Oh theres a brain all right. It sjust that the brain is madeout of meat That s what , 've been " . I trying to tell you " " So . . . what doesthe thinking? ' ' " Youre not ' , understandingare you? Youre refusingto deal with what I m telling . . Thebrain doesthe thinking. Themeat" you ' " !" ! Thinkingmeat Youre askingme to believein thinking meat

422

10 Chapter
" Yes ! meat Loving meat Dreamingmeat Themeatis the ! . . , thinking meat Conscious wholedeal! Are you beginningto get thepicture or do I haveto start all over?" - Terry Bisson Nebula Award Nominee from "They ' re Made out of Meat" (OMNI , , of . , ) Magazine April 1991 . (Reprintedwith the permission Mr. Bisson)

Chapter Objectives I.To the and and mind a robotic explore ramifications particular to unusual ,control withimmune ,concept of, regardconsciousness homeostasis thought ,emotion . imagination 2To and of . consider . robotic ,theaspects including systems - equivalence nanotechnology , . entertain 3To notionrobot . of human
In this chapter we moverelatively far from mainstream roboticsandprobeits , areasQuite often, practicingroboticistsbecome . enmeshed pragmatic in fringe issuesand ignore their work' s philosophical ethical, and evenmetaphysical , ramifications We now explore some of the deeperquestionsregardingthe . potential for robotic intelligence and in so doing discovera wide range of views on the subjectsof robot mind and body. Certainly interestingsubjects , often controversial with nothing broadly agreedupon: Intelligence mayor , ; may not be achievable computation robotsmayor may not be ableto attain by consciousness roles, if any thatemotionsandimaginationplay in artificial ; the , ; , systemswhat otherbiological models suchashormonalandimmunecontrol , haveto offer; whatof incredibly smallnanorobotsandin what sense ; , systems if any a robot canbe viewedasequivalent or a successor humanbeings to to . ,

10 ISSUES TIlE ROBOT .1 MIND OF

The idea of artificial minds has beena central philosophicalquestionin AI research sinceits inception Attributing a mind to a robot is viewedby manyas . a substantial . leapof faith for a scientist The very conceptmay be disturbingto robotsaspotentialcompetitors all at manyof us, for it canraisefearsregarding levelsof humanendeavorCanrobotsthink? Canrobotsbe conscious selfaware . or ? Can they feel or dreamas we do? It is certainly easy at this point, to , dismissthesequestionsIt is useful however to examinea broadperspective . , , of theseissues often lessscientificandmorephilosophicalthanwhat we have , becomeaccustomed , but nonetheless to worthy of explorationwith an open mind.

423

: , Fringe Robotics BeyondBehavior

10 .1.1 On Computational Thought

We begin with a caveat This sectionis not intendedas a discourseon phi: losophybut rathera brief review of severalnotableopinionson this and other questionsregardingmind as it pertainsto machinesin generaland robots in . particular Should a machinebe said to think if it can fool a human who is merely -based of ? observingit into believingit is capable thought This performance approach is the basisfor the classicThring Testof machineintelligence in which , the definition of thoughtis basednot on machineconsciousness ratheron but humanfallibility (Thring 1950 Epstein 1992 . This definition by deception ; ) carriesenoughweight to prompt the annualsponsoring the LoebnerCompetition of , which offers a $ 100000 prize to a machinethat can success pass , fully the Thring Test .

Those for whom the proof of possession of thought is based solely upon observed action have little reason to read further . For those who believe that a thinking agent must possessmore than merely the ability to exhibit plausible actions for a wide range of situations in order to be considered able to think , we proceed. Bellman ( 1978) states that since no one knows what " think " really means anyway, we cannot fairly answer the question of whether a machine can truly think . He argues from a mathematical perspective that computers can perform processes representative of human thought (i .e., decision making and learning ), but since no one can precisely define what constitutes thinking , the question cannot be rigorously answered.

A different stanceexpressed Weizenbaum echoedby Albus ( 1981 and , by :" humansthey would have to p. 297) states For robots to truly understand be indistinguishable from humansin bodily appearance well as physical as and mental development and remain so throughouta life cycle identical of humans This leadsto the strong conclusionthat robots will never be able ."

and

two

of other . the

respondents

front Test

computer

in and the the to Turing sitting the computer between a questions person is

a passed any ask which have to

to

involves of discriminate free

said

one

Test is , is cannot tester Turing

terminals

the computer

of questioner the the , of the intelligence If

version end subject This . being other popular human the

One terminals on human the

424

10 Chapter
' to comprehend humans values and thus never be able to think as humans , do. Brooks ( 1991 p. 22), on the other hand arguesthat these aspectsof , , are a naturally occurring by-product of a behaviorbasedeffort: intelligence " will not needto be programmedin. They will Thought and consciousness . ." emerge can easily be characterized AI ' s most ardent opponent as . Roger Penrose He deniesthe possibility that computational escaneverleadto thought process andpresents four alternate on of perspectives the issues thinking andawareness 1994 : ( penrose ) 1. All thinking is computation thus computersare capableof thought (This . , is often referredto asStrongAI ). position 2. Thoughtis a result of the brain' s physicalactions Computers simulate . can this action but a simulationis neverthe sameasthe thing simulated andthus , , cannotthink. computers 3. The brain' s actionscannotevenbe simulatedcomputationally . 4. Awareness cannot be explained by any scientific approachand thus is unattainable . computationally 's NewMind 1989 Penrose In TheEmperor ( ), presents strongarguments against and on computational intelligencebased pillars of mathematics computational 's Theorem and the Church Turing Thesis , , theory GOedel Incompleteness ' s books for the full . others The readeris referredto Penrose among development of his dismissalof AI as a meansfor producingthought consciousness , , . andself-awaremachines 's within the AI Obviously Penrose point of view has met stiff resistance . ) community. Most (e.g., Brooks and Stein 1994 point out that his arguments are fundamentallyflawedand havebeencontradicted earlier mathematical by : , ( ) investigationsArbib 1964 . Caustically Brooks states " . . . Penrose. . not error . the only makes same ThringGOedeI , but thenin a desperate the of to of methodology attempt find theessence mindandapplying standard to to , namely finda simplifying , mystical underlying principleresorts analmost physics " mechanics and reliance quantum on , (Brooks Stein1994p. 23). a book, Nonetheless devotes majorportionof his second , Penrose steadfastly Shadows / the Mind ( 1994 to disputing the plethoraof arguments o ), against . his position dismissingsomefar more easily than others He statesthat intelligence , and understanding , , requiresawarenessan requiresunderstanding in holding out for the . Most puzzling is his optimism aspectof consciousness intelligencebasedon a new science possibility of a type of noncomputational

425

: Fringe Robotics BeyonqBehavior

of quantumphysics He contendsthat intelligence is a consequence this . of sort of activity naturallymanifested within the microtubules locatedwithin the brain' s neuronsMuch of this argument speculativebut nonetheless . is , inbiguin ing, especiallygiventhe recentadvances the theoryof quantum computation ; (Lloyd 1996 Hogg 1996 . )

areTh dev tha on th sc Quant comp curre hypo ofonop atoms .Lin)dis tpro he wa na of (.the They explo quan para state of.qu asta set ofan ,capa )logic particu existi supe ab endow them with theha tosim doca clas do ever more .class Scom 199 and ho (ofof using quant Fact are two (1996 )exam loyd compu prob pro inh efficie soluti this ofto mac
for the issues surrounding

on , Despitetheragingdebate the very possibility of thinking machinesthere is demonstrable valuein pursuingtheir developmentThe success achieved . es in behaviorbasedrobotic systems generallydo not lay claim to humanlevel . , intelligence nor do they arguethat the systemscreatedare aware To most roboticsresearchers is an irrelevantquestion their goal beingto build useful this , and, at the very least debatablyintelligent machinesThe commonlyheld . , belief that engineers scientists and havesuccess fully proventhat a bumblebee
is incapable of flight may also have implications philosophers and thinking robots .

10.1.2

On Consciousness

the of Chalmers 1995 p. 81) characterizes mostmysterious , ( aspect consciousness " es as" how physicalprocess in the brain give riseto subjective , experience i .e., the experiences color, pain, emotion and feelings in general Some of . , . that they have unshrouded this mystery In Consciousness philosophersboldly argue , , , , Explained Dennett( 1991 p. 433) arguesusingreductionism that " all that to complicatedslewof activity in the brain amounts conscious experience assume the brain canbe viewedasan information that ." His conclusions a computerif you will , from which he further concludes , processing system . can that softwarebasedcomputersystems give rise to consciousness Perhapsthe single most disputedthought experimentused to deflate the ' is ) potentialof machineconsciousness Searles ChineseRoom ( 1980 :

426

Chapter

Searle ultimately assertsthat no robot could ever be conscious( Boden 's 1995 . Just as Penrose arguments met with widespread disclaim so did the , ) ChineseRoom refutation of strong AI . Dennettcountersthat thesetypes of ' ' " the thoughtexperiments work preciselybecause they dissuade readerfrom " , trying to imagine in detail, how softwarecould accomplishthis (Dennett 1991 p. 435). , The believersin computational consciousness . ) persist McCarthy ( 1995 believes is consciousness not only possiblefor robots but necessaryThis consciousness . , ' is intendedto be different from a humans and accordingto Mc, , , Carthy should not have humanlikeemotions in order to make robots more servile A robot consciousness . , requiresthe ability to observe manythings including its own physicalbody theextentof its knowledge what it doesor does , ( not know), its goalsandintentions the history or basisfor its beliefs, andwhat , it is capableof achieving . into Bellman ( 1978 p. 94) attemptsto draw mathematics the fray with the , cannotbe made observation that although"the generalareaof consciousness " ..." of consciousness be treatedby mathematical can many aspects precise means which meansthat we can have a computerbe consciousin certain , as . ." of ways The result is a characterization consciousness a control process will Moravec (forthcoming statesthat a robot' s consciousness eventually ) exceedthat of people " Someconfigurationswill make a robot more thoroughly : than the average human . . . " This view is consistent . with conscious ' his long-time articulated expectationthat robots are humans natural suc

An.u tofb wh he no ot is lo in a (,bla CP )the ,cu Eng spe sp la rooh with aca deh m as r (th )a writ osip u pa tare tha he wh tod do w (has )slip pro say pr Chi cha .Me AC Ch ch tm () pa co ato sm in do .hu Th u ru pre thro op toby tran intth Ch fir c fr ,as per the top en th ru bo an inp slip cor b out ins ne T bo spe cr an pro gen in q The the wr for ou (th )an pas oc sma .sys The ca be su th ope fu ifun aof tak the the hu .co Se Tes ,not com pla tran no wh th of und sli pa not und .co Ch wi up rig b doe con pos

427

: FringeRobotics BeyondBehavior cessors These machines constitute the next logical step in evolution , our . " mind children " , ultimately capable of transcending human biological frailty ( Moravec 1988) . Consciousness may be overrated anyway. Minsky ( 1986, p . 29) argues " In ' " general, we re least aware of what our minds do best and that consciousness arises when our automatic systems begin to fail . Moravec ( 1988, p . 44), despite his own tendencies to the contrary , observes that " robotics research is too practical to seriously set itself the explicit goal of producing machines with such nebulous and controversial characteristics as emotion and consciousness" . Most roboticists are more than happy to leave these debates on consciousness to those with more philosophical leanings.

10 .1.3 On Emotions
Severalroboticists however recently have paid attentionto emotional state , , andits impacton behavior Wecanintuitively understand emotionsindeed . that influencebehavior When someone angry they generallybehave . is , differently than when they are happy The effect may be anywherefrom subtleto quite . . , strong dependingupon the individual and the strengthof the emotion But what is this emotionalstuff, andwhy would it be of possibleimportance use or to a robot? We have alreadyseenexamplesof the attribution of emotion to behavior basedsystemsBraitenberg in particular unabashedly . describes vehicles his , , as possessing , love, and aggressionas discussed section1.2.1). Albus fear in ( :" , ( 1981 p. 208) states Emotionsplaya crucial role in the selectionof behavior ." Though lower life forms exhibit simple emotions(pleasureor pain), humansapparentlyhavea much broaderrange(hate love, anger fear, happi , , ness disgust amongothers . Neurologically it is generallyacknowledged that , , , ) humanemotionoriginateswithin the brain' s limbic system . ' U.S. Supreme CourtJustice JohnPaulStevensfamous Modifying Associate ' t defineemotion but we know it when we seeit. And , , quotation we can perhaps . that is the crux of the argument that robotscanindeedpossess emotions fear Moravec( 1988 contends a robot is actuallyexperiencing when upon that , ) a of encountering stairwell, it backsaway from the dangerbecause adetectcliff sensingsystemcoupledwith a deal with-cliff action system Is emotion . thenin theeyeof theobserver For eachof us, speaking ourselveswe would ? of , , , clearly answerthat it certainly is not. We know when we are angry happy or whatever independent externalobservationBut we ascribethis emotional of . , . capacityto othersprimarily throughobservationSo how do we know whether a robot is or is not experiencing emotion ?

428

Chapter10

:" , ) Minsky ( 1986 p. 163 states The questionis not whetherintelligent machines can haveany emotions but whethermachines be intelligent without can , ." any emotions Emotional capacitymay better adaptrobots to deal with the world: Love can provide social behavior useful when cooperatingwith other agents(humanor robotic); angercan be useful when competingwith other agents andpain or pleasure be usedfor reinforcement can ; learningand self-protection As Moravec(forthcoming observes" In general robots will . : , ) exhibit someof theemotionsfound in animalsandhumans because thoseemotions areaneffectiveway to dealwith the contingencies life in the wide, wild of world." To date Japanese scientists haveconducted mostof the pragmaticresearch , on giving robotsemotions Frustration an emotionnot uncommonto robotics . , researchers as at , oftenserves thebasisfor emotionalcontrol. Research Nagoya Mochida et al. 1995 has looked at robots that can experience ) University ( two states pleasantness unpleasantness variablerepresenting : and .A frustration the : , represents states low frustrationis pleasant but high frustrationis to the contrary Simulationshavebeenconducted . usinga Braitenbergstyle architecture ' with a neural emotionalmodel that alters the systems supplemented behavioras it becomes more or lessfrustrated This enables to escape . it , traps suchas box canyons which it cannotaccomplishwithout this emotionalbehavioral , switching. Research MITI in Japan(Shibata Ohkawa and Tanie 1996 extendsthe at , , ) use of frustration to multirobot systems In the contextof empty can collection . ' s own behaviorbut also tasks frustration arisesnot only from the agent , ' from otherteammembersperformanceThe frustrationlevel altersthe action . selectionprocess appearing result in greatercooperation to than would occur , otherwise Section 10.2.1 describes actual vacuumcleaningrobot, Sozzy . an , that alsousesemotionsas a basisfor action selection . Of course the debatewill continueasto whethertheserobotsreally experience , emotion An importantpoint to takefrom this discussionhowever is that . , , havesomeutility in the contextof emotionalcontrol systems biological may behaviorbasedrobotics evenif they servemerely to inspire modelsthat are , quite limited or ratherfar afield.

10 .1.4 On Imagination
What of imagination " Imaginationgives us the ability to think about what ? " we aregoing to do beforecommitting ourselves action (Albus 1981 . This to ) for simulation (imaging future actions providespotentially useful ) capacity

429

: FringeRobotics BeyondBehavior

.1 Figure10 added above beyond and . Highlighted are Meta 's architecture Toto components those ' . Totos corecapabilities . of feedbackas to the utility andrelevance any plansunderconsiderationThe is quality of feedback directly relatedto the quality of the simulationitself and aboutthe world. the accuracy its underlyingassumptions of At MIT , imagination in at least one sense has been integratedinto the , , architecture(Stein 1994 . Cognition, viewed here as high-level ) subsumption or deliberativereasoning planning is treatedas imaginedinteractionwith the , world. This cognitivesystemis not disjoint from the robot control architecture , as is the casein many hybrid architectures chapter6), but rather usesthe ( as underlyingbehavioralarchitecture the simulatoritself. in wasembodied MetaToto, extendingthe earliernavigational This novel approach architecture Toto of within the subsumption work usingrepresentation Toto' s . section5.2.2 1). Figure 10.1 depictstherobot' s controller which reuses , ( in architecture its entirety wrappingthe imagining simulatoraroundit. Sensing , scan is imaginedusingvery simplesonarmodelsanda straightforward -line : algorithm. Acting is imaginedby the updatingof threepositionalvariables x-position, and heading In essencethis is a rudimentarysimulator . . , positiony of MetaToto, however is capable exploringfloor plan drawingsandimagining , environmentswhereas how it would movein thesepreviouslyunexplored , in Totocould not. The actualsonarandcompass readings therobot architecture

430

10 Chapter havebeenreplaced with imaginedones Although simulators . that usethe same control codeasactualrobotsarenot new (e.g., Mission , section9.9.2), the Lab is . cognitiveframeworkin which this work is couched interesting Whetherthis is truly an exampleof robot imaginationis subjectto debatejust as were the , notionsof robotic thought consciousness emotiondiscussed the preceding in , , and sections .

10.2 ISSUESOF THE ROBOT BODY

We now examine some fringe areas not directly related to the debate regarding the robot mind . Nonneural control systems have the potential for contributing to intelligent robotic systems. In animals, the endocrine system uses chemical messagesfor autoregulatory purposes whereas the immune system responds to external events in a defensive manner. This section explores the implications of these alternate control paradigms for robotics , then examines the issue of scale: What if robots could be made to operate at a molecular level? This draws us into the field of nanotechnology, currently little more than a dream but of potentially great importance .

10 .2.1 Honnones Homeostasis and

The endocrinesystemin mammals using hormones chemicalmessengers as , , servesas a meansfor both information processing and control. This homeo static control systemis concerned with maintaininga safeand stableinternal . is , operatingenvironmentwhetherfor animalor machine Homeostasis a term to biological systems the process which that safeand for typically applied by stablestateis achieved maintained and . ' " Gerald( 1981 describes endocrine the ) systems basicrole asfollows: [The endocrinesystem may be comparedto an orchestra in which, when one , ] instrumentis out of tune, a perfectensemble impossible . . . It is constantly is . the internal environment and it is ideally situatedto function in , monitoring " to , response psychicstimuli. (Boyd 1971 p. 421). A basic biological function of the endocrinesystemis to maintain the or' (homeostasisThis is accomplished the ). ganisms internal self-consistency by ' endocrine ductless glands directly secreting their chemicalmessengers ( ) (hormones . ) into the bloodstreamThe circulatory systemthen carriesthesemessengers ' , broadcaststo all the organisms cells. Different cellshave (in essence ) to secretionssomereactingmarkedlyto certain , differing responses endocrine

431

: FringeRobotics BeyondB.ehavior

hormones othersnot at all. Selectivetissues called target tissues are selec , , , . tively aroused a specifichormone An exampleis a hormonesecreted the by by that targetsbonetissuespecifically Somehormones nonspecific . are thyroid gland , suchasinsulin, which actson almostall cells (with the notableexception of mostbrain cells), affeCting their energy(glucose uptake . ) feedbackmechanisms fundamental endocrinesystemcontrol are to Negative . Onetypical example( Boyd 1971 illustrateshow the hypothalamus monitors ) the releaseof severaldifferent hormonesand coordinatestheir effects basedon central nervoussysteminputs. By applying this negativefeedback control regime the system whenstressedcanbe restored a steadystate an to , , , , crucial to the process homeostasis of . ability The biological endocrinesystemis concernedwith three different areas : function, andmetabolism , , . growthanddevelopmentnervoussystem regulation with the latter perhapspaying the highest dividendsin robotics Lehninger . as" , ( 1975 p. 363) definesmetabolism a highly coordinated purposefulactivity in which many setsof interrelated. . . systems , participate exchangingboth matter and energy betweenthe cell and its environment This is resource ." at we management a very low level. The questionis what resources should be concerned with in the roboticsdomain . ' . is Energymanagement onechoice Justasglucosefuels most of the body s cells, someform of energymustbe madeavailablefor the robot. In mammals , two typical modes glucosemetabolism found, "feast or famine." Cellular of are es . process arealwaysdrawingenergyfrom their environmentWhenglucoseis abundant the blood, insulin is released signalingto the cells that their energy in , . uptakecanbe increasedIf the glucoselevel dropsoff , the hormonelevel . also drops signifying a fasting state This hormonalreleasealso affectsbiological , tissuesmarkedly in additi<n to single cells, by bringing entire , organ ? into on subsystems either high or low statesof activity depending the organ ism' s current state This aspectof energymanagement highly suitablefor . is control subsystem . incorporationinto a robotic homeostatic . Warm bloodedanimals Anotherpotentialapplicationis thatof thermoregulation must maintainconstanttemperatures throughouttheir lifetimes. Robots suitableoperating do not live within such rigid constraints but nonetheless , to . rangesmust be adhered if we expectreliable performanceRobotsare expected to perform in more hostile environments than people(e.g., space and ) . must havethe ability to regulateinternal temperatureIt is not proposedthat to the cooling andheatingmechanisms robotsbe analogous thoseof mammals for is . , ; thatwould beabsurdWhatever temperature management system used : however two typesof stress canbe anticipated es ,

432

Chapter10

. Global stress Heat must be exchanged : betweenthe robot and its surroundings to restoreacceptable conditions(e.g., a robot on the sunnyside of Mercury ). . Local stress Heatmustbe redistributed : within the robot to maintainreliable . could operationof a particular subsystemThe failure of a single subsystem have the domino effect, ultimately resulting in the robot' s completefailure. An exampleof local stresswould be the overheatingof a robot' s arm while . servicinga furnace In either case temperature must be regulatedby a control system Sincethis . , occursunconsciously mammals(global stressthroughsweating in regulation or panting local stressby dilation or constrictionof blood vessels, homeo , ) staticcontrol caneffectively manage function. this notification and resultantbehavioralparameteralterationscan Emergency also be carried out quickly and efficiently using a broadcast communication mechanism This loosely parallels the secretionof epinephrine(adrenaline . ) ' that markedly and rapidly increases rate of a creatures process in response the es to an unanticipatedevent The fact that dormancycan be induced . over the samechannels should not be overlooked Indeedmuch of . , rapidly, the " fight or flight" response be embedded this manner can in .
10. 2.1.1 The Homeostat

in ) Ashby ( 1952 was amongthe first to developthe notion of homeostasis a . , cyberneticcontext extendingthe principles of biology to machines In particular is , he arguedthat adaptation essentialand that it is achieved maintaining by certainessentialstatevariableswithin acceptable physiologicallimits , blood glucoselevel maintenance thermoregulation two examples and as , citing ' . he to amongothers Because sawmaintainingstability asessential a systems survival he createdan unusualdevicecalled a homeostat embodies that these , . It consistsof four interactingunits eachcontainingan electromagnet principles and a water basedpotentiometer(figure 10.2). The units are fully interconnected : eachreceivesinputs from and eachsendsoutputsto the others In . testsof the system certainsettings stablebehavior(with the magnets , produced othersettings ) movingto a centralpositionandresistingdisplacementwhereas ' with the magnets velocities increasinguncon yielded runaway instability ( devicegiven the complexityof . Although a seeminglyuninteresting trollably) ' , today s robots thehomeostat provideda testbedfor the notionsof homeostatic in . stability in machines the 1950s

433

: Fringe Robotics BeyondBehavior

(A)

(B)

.2 Figure10 ' :( device(8) thecircuitfor a single unit. ; Ashbys homeostatA) theactual

.L I:A:P " i '- : ; ~ =:~i:l~jA ;~=~ :~~ i :~ ~: r ::::J Z~=:~ :,:~t= B ~ i~~c ~3 I~~ ~=:: ~~ (]
Mr ) : d

~ z ~ F8 ~ ~D I~ ~
I

434

10 Chapter -Based 10 .2.1.2 Schema Homeostatic Control

The addition of a new classof behavioralconttol units called signal schemas (Arkin 1988 provides a meansfor a robot to senseand transmit to motor ) behaviorsinformation regardingits own internal state Thesesignal schemas . are of two types transmitterschemasassociated : with specific internal sensors , within motor schemasThe honnonal . , and receptorschemasembedded , of targetability is achievedby allowing the transmitterschemas to concept broadcast their informationto all activebehaviors Only thosemotor schemas . whoseactivity is dependent a particulartypeof informationcontainreceptor on schemas sensitive thosespecificbroadcast to . messages Transmitterschemas sendinformation pertaining to one particular aspect of the robot' s internal state Their role is ~ provide the feedbackrequiredto . o achieve homeostatic control. For example a sensor measure robot' s available can a , -poweredvehicles this might involve fuel reservesIn the caseof battery . , an ammeter for petroleumpoweredvehicles a fuel tank measuringdevice ; , could be used The rate of consumption also be monitored providing additional . can , informationfor negative feedback . analysis within the motor schemasprovidethe mechanism , , Receptorschemasembedded for modulatingthe motor behavioritself. In response the information to the transmitterschema broadcaststhe receptorschema altersparameters , within its motor schemachangingits output. If fuel reserves runninglow, are , motor ratesare tunedto run at more efficient levels Internalchanges produce . can shorter albeit more risky, pathswhen fuel depletionwarrantsthe risks. , The unit hasonereceptorschema eachtransmitterto which it is sensitive for , the conceptof targetabilityby specifyingwhich, if any of the , implementing transmittedsignalsthe behavioralcontroller shouldbe awareof. Figure 10.3 . depictstheserelationships In the caseof energymanagement transmittermessage emanates from an ,a internal sensorreporting availablefuel reservesSincethis message transmitted . is globally, it affectsall targetedmotor behaviorsuniformly. In the case smooth more efficient (albeit slower motion of energyreduction this produces , , ) the . In thermoregulationdecreasing rate of motion reducesthe amount , of heatproducedper unit of time, allowing the motorsto dissipateheatmore effectivelyandusepowermoreefficiently. As an example figure 10.4 showsthe effectson navigationas the robot' s , . initial fuel , reserves rangefrom full to almost empty As the energysupplies dwindle, the robot comescloserandcloserto the obstaclesmovingat a slower , . and more efficient speed Eventually the courseof the path taken actually ,

435

Fringe Robotics : Beyond Behavior

Figure 10.3 -based Homeostatic schema control architectureThe highlightedcomponents pertain to . homeostatic control. switch es, producing a much shorter path (in terms of distance but not time ) in reaction to low fuel conditions . This set of paths clearly indicates the impact of available fuel reserves on the schema-based navigation process. Additional results, including those concerning thermoregulation , appear in Arkin 1992c.

10 .2.1.3

Subsumption -Based Hormonal Control

Another application of the notion of honnonal control involves the development of a honnone - driven autonomous vacuum cleaner named " Sozzy" (figure 10.5) ( Yamamoto 1993) . This system is really a mixed metaphor in which ' " " honnonalanalogies are used to modify the robot s emotions , specifically fatigue , sadness desperation, and joy . Figure 10.6 shows the means by which , an emotion - suppressing behavior is added to an underlying subsumption- style

IS1

Sensors

MOTORS

ROBOT

Sensor

Internal

Subschema

Schema

Schema

Schema

Sensors

Schema

Perceptual

Motor

Receptor

Perceptual

Environmental

Transmitter

Internal

IS

PS

PSS

MS

ES

RS

TS

Key

MOTORSCHEMAS Al T N ME . . . RON I SENSORS V E ~ R N E M N 0 T EN

436

Chapter10

....

*
Available

*
*
START

.t*

Figure 10.4 become . Collection of padisreflectingdifferent fuel reservesNote as the fuel reserves the pathsget closerandcloserto the obstaclesfinally resultingin a complete , , depleted ' from detouringto the upper in quality asit eventuallychanges change the path s general . at regionsto moving moreclosely aroundthe obstacles slowerspeeds

. control systemthat modifies the underlying active behavioralconstituency Hormonal state variablesthat reflect the various emotional statesare maintained both . behavior A function that receives within the emotion suppressing internal(e.g., batterylevel, time expired andexternalstimuli (e.g., lossof beacon ) . thesestates The net result is behavioralswitching as opposed ) regulates -basedhormonalcontroller . to behavioralmodification as seenin the schema in a laboratorysettingwherethe A robot implementingthe systemwastested roseandfell over to hormonelevelscorresponding the variousemotionalstates ' the time. This changed robot s overallbehavior giving it the subjective , appearance " of being " more friendly andmore lively (Yamamoto1993 p. 221) than , . whenthe hormonalsystemwasinactive

/ , Fuel Least

437

: Fringe Robotics BeyondBehavior

10 Figure .5
: -driven robot. (Photograph .) Sozzy A honnone courtesyof MasakiYamamoto

S E N S 0 R S

A C T U A T 0 R S

Figure 10.6 Honnonal behavioral switching Inhibition from the emotion suppressing . behavior the robot inactive until an emotion is enabled which in turn inhibits the inhibition , keeps , effectively selectinga suitablebehavioralset.

438
10. 2.2

10 Chapter
Immune Systems

Another biological control systemparallel exploredin the contextof robotics involvesthe immune responseImmune networksdetectand attack antigens . . (alien nonself materialssuchas bacteria by producingantibodies The bone ) marrow and thymus gland createlymphocytesof varioustypes that regulate the production of antibodies and circulate throughout the lymphatic system in mammals Specificantibodiesattack specificantigensby recognizing . , antigenicdetenninantsfonning a complexcontrol systemcapable particular of recognizingandeliminatingboth previouslyencountered new antigens and . In Japan researchers haveappliedprinciplesinspiredby immunenetworks , to robotic control problems Using immune systemmodelsinitially derived . for fault tolerance( Mizessyn Ishida 1993 the simulatedlearningof gait and ), for a six-leggedrobot hasbeenachieved , , (Ishiguro Ichikawa and acquisition Uchikawa 1994 . Immunesystemmodelshavebeenextended includemultiagent to ) robotic systems(Mitsumoto et al. 1996 . Here parallelsare drawn at ) : severallevels The robot and its environmentare modeledas a stimulating can , antibody antigenrelationship and robot-robot interactions be both stimulating and suppressinganalogous antibody antibodyrelations . Eachrobot to ( ) decidesits next action basedon theserelationships with other robots and the world, organizingitself to effectively conductthe task. The systemhasbeen testedin simulationonly on a foragingtask, with an eyetowardmovingit onto a six-agentmicrorobotcolony . Both of theseexamplesare basedloosely on the idiotype network model are (Jerne1973 in which novel, randomlycreatedantibodies initially treated ), stimulationor suppression other antibodies of as antigens resultingin systemwide , . of in steadystateconditions The presence other , eventuallyresulting material can disrupt this equilibrium and as in homeostasisthe , , antigenic systemthen respondsin a mannerto restoreor achievea new steadystate modelsas condition. Others(e.g., Bersini 1992 haveusedimmuneresponse ) methodssimilar to Q-learning(section an inspirationfor reinforcement learning . 8.3), but this hasasyet not beenappliedto actualrobot control systems

10 .2.3 Nanotechnology
What if we could build really small robots, robots so small that they could operate at the molecular level? As fantastic sounding as it is , this is the domain of nanotechnology, where machines operate on atomic scales. It is not as absurd as it first sounds. One could argue that nanomachines already exist

439

Robotics : Beyond Behavior Fringe

operating within cells. These however result from natural sources. Protein enzymes routinely are involved in the assembly and disassembly of the stuff of which we are made. Is it really so implausible that engineered machines

? could be devisedto serv similar purposes ~ has as Drexler' s vision of nanotechnology served the foundationof the field. In Enginesof Creation( 1986 Drexler describes future in which molecular a ) machines commonplaceCell repair machinesfor example havethe ability are . , , to curediseasereverse infection. He , , aging or serveasactiveshieldsagainst of . alsoraisesthe spectre their beingusedfor evil endsor destructive purposes Drexler' s subsequent book Nanosystems1992 providesmore of a scientific ( ) basisfor his earliervision, describing techniques which molecularman the by has . ufacturingcouldpotentiallybe achievedMolecularmanufacturing morein in of commonwith biochemistrythan with engineering tenD S precision control , defectrate productsize andcycle time. , , , as Moravec ( 1988 envisionsnanotechnology the meansby which robots ) . could truly becomeself-assemblingFor example a robot bush (figure 10.7) , could self-constructand would have a structureunlike anything we' ve seen could change thusfar. Roboticcilia would propelthe bushabout andits shape , the control. . dynamically Local reflexes might handlemuchof , Nothing closeto an actualnanorobothas yet beenproduced but many researchers . are still working on a very small scale This is the domainof microrobots that differenttasksin , very smallrobotic systems cando fundamentally . different ways than the more conventionalsystemswe havealreadystudied the On thesescalesfriction becomes dominantforce ratherthangravity. Gnat , robotshavebeenproposed ) (Flynn 1987 andbuilt (Flynn et al. 1989 that can ) : fit on a single electronicchip. Thesesystemshavenumerouspotential uses autonomous billboards for the CIA , multiagentswarmsfor space , exploration bugs from a , , eyemicrosurgeryandpatchingholesin or removingbarnacles ' s hull . ( Flynn, Brooks, andTavrow 1989) ship : several microrobotsystems MIT ' s AI Lab hasdeveloped . Squirt: a low-cost prototypemicrorobotbuilt for roboticseducation(Flynn et al. 1989 . ) . The Ants: a colony of microrobotsbeing developed explosiveordnance for 10.8) . disposalapplications(figure . The Rockettesa colony of microrobotson the orderof 10gramseachbeing : . for developed planetaryexploration thus far is the hybrid Perhapsthe most unusualmicrorobot encountered at insectrobot Takeuchi( 1996 has developed the University of Tokyo. This ) : systemis part robot and part cockroach two severedcockroachlegs serve

440

10 Chapter

Figure 10.7 A self-assembling . robot bush ( Figurecourtesyof HansMoravec) .

as the actuatorsfor a single chip microcontroller It is able to walk when an . artificial body is attached the legs. This roboroachis reportedlycapableof to insertedinto functioning for approximatelyone hour andusesfour electrodes the cockroach . legs

10 ONEQUIVALENCE BETTER .3 (OR )

We now briefly examinesomeissuessurrounding ultimaterelationshipbetween the robotsandhumansWill robotsinherit theearth Will humans . ? ultimately residein robotic form? This sectiondiscuss the positionsof thosewho believe es either (or both) of thosethings will happen Although it may be easyto . -mindedlyexplorethesepositions thesepoints of view, let us open . disregard

441

: FringeRobotics BeyondBehavior

.8 Figure10 B Ant microrobots . (photograph of courtesy Rodney rooks.) " , , ) Minsky ( 1994 p. 109 respondsto the question Will robots inherit the " : " Yes as we bodiesand brainsusing nanotechnology earth ? , engineerreplacement . We will thenlive longer possess , greaterwisdomandenjoy capabilities as yet unimagined Minsky' s answeralso toucheson the issueof humansas ." from robots Accordingto this view, thetranscendence biology to technologyis . ratherthanfear. We will ultimatelybe freedfrom our bia cause celebration for . ologicallimitations. Eventhe option of immortality is posed Humansbecome - ultimately there is no distinction. This somewhat machinesand vice versa radicalviewpoint flies in the faceof the counterarguments againstmachineintelligence earlier in this chapter we , thought and consciousness encountered , . but we continueour explorationnonetheless Moravec( 1988 arguesthat this is our destiny that, as statedearlier these , , ) future robotsareour mind children. But how do we becomeour robotic equivalent ' ? This processof a persons becominga machineis referredto as transmigration , which might occurin severalways(accordingto Moravec : ) -fidelity surgicalneuronby-neuronreplacement 1. One approach might involve a high . of your brain with an electronicneuroncounterpartAs this would be a step by-step processconductedwith verifying simulationsat every replacement , step at what point do you stopbeing a humanandbecomea robot?

442

Chapter10

2. Anotherstrategy would involvea high-resolutionbrain scanthat, in a single would createa new you " while you wait." , operation 3. Perhaps computerthat you would wear throughoutyour lifetime would a recordall of your life experiencesOnceit learnedwhat it was like to be you . andcould act equivalentlyto you, the recordcould be transferred a machine to substrate . 4. Another approachwould be to severthe CQrpus callosum which connects , the brain' s two hemispheresand attacheach end to a computerthat at first , the . passes messages through while also recordingthem Eventually your biological brain would die, but during your lifetime this computerwould have learnedhow to be you andcould continuein that capacityindefinitely . The net result, accordingto this view, is that the sum total of you is a of : life programindependent wheretheprogramresidesin carbonbased forms, in silicon, or perhapsin somethingelse You could run (think?) at speeds . millions of timesfasterthanthe limitations biology imposes you. It matters on not if your frail biological life form dies you still exist: " If the machine , ] you inhabit is fatally clobbered the [ backup tape can be read into a blank sincethe copy With . , resultingin anotheryou, minusthe experiences computer deathwould be very unlikely" (Moravec 1985 p. , , enoughcopies permanent 145 . ) . Fringe robotics indeed For this point of view Moravechasbeenlabeleda " DNA traitor" by many who either fear or dismissthe consequences these of . thoughts

10 OPPORTUNITIES .4
In concludingthis book, it might be wise to provide someguidanceto those who follow by describingwhat problemsremainto be solvedto advance the scienceof robotics Many openquestions . warrantfurther investigation Justa . few arementioned below: . Identifying ecological niches where robots can success fully competeand survive making them sufficiently adaptable changes the world they inhabit to in , . . Increasingaccessibility bringing robotics to the masses : through suitable interface specification andprogrammingsystems . , , . Representing controlling sensingby viewing it as a form of dynamic and communication . agentenvironment

443

: FringeRobotics BeyondBehavior . Improving perception : new sensors selective attention mechanisms, gaze , control and stabilization , improved eye-hand coordination , foveal vision , and specialized hardware, among others. . Further exploitation of expectations, attention , and intention in extracting information about the world . . Understanding more deeply the relationship between deliberation and reaction , leading to more effective and adaptive interfaces for hybrid architectures. . Evaluating , benchmarking , and developing metrics: In order to be more accurately characterized as a science, robotics needs more effective means for evaluating its experiments. Although progress is beginning to be made in this area (Gat 1995) much more remains to be done. . Satisfying the need for far more advanced learning and adaptation capabili . ties than are currently available. . Creating large societies of multiagent robots capable of conducting complex tasks in dynamic environments. . Using robots as instruments to advance the understanding of animal and human intelligence by embedding biological models of ever-increasing complexity in actual robotic hardware. ' Certainly , the roboticist s plate is full of a myriad of important and exciting problems to explore .

10 CHAPTER .5 SUMMARY
. The issue of whether or not robots are capableof intelligent thought or consciousness quite controversialwith a broadspectrum opinion ranging is of , from " absolutelynot" to " most assuredly so." . Robotic emotionsmay playa useful role in the control of behaviorbased . , systemsalthoughtheir role is just beginningto be explored . Homeostatic control, concerned with managinga robot' s internal environment , canalsobe usefulfor modulatingongoingbehaviorto assistin survival. . Immunesystems alsobeginningto beexploredasa means controlling are for both individual andgrouprobot behavior . . Nanorobotsand microrobotscan revolutionizethe way in which we think aboutrobotic applications . ' . Oneschoolof thoughtin roboticsasserts thesemachines mankinds that are naturalsuccessors . . As in any importantendeavortherearea wide rangeof questions , waiting to be answered well asopportunities be explored as to .

References

Aboaf, E., Drucker S., andAtkeson C. 1989 "Task-Level RobotLearning Jugglinga . : , , TennisBall More Accurately Proceedings theInternationalConference Robotics on ," of and Automation ScottsdaleAZ , pp. 1290 95. , ,

." Affordances Slopes of : , , , Adolph K., GibsonE. J., andEpplerMiA. 1990 Perceiving ' Locomotion " TheUpsandDowns Toddlers of #16 , Emory Cognition Project Report , of . , Emory Deparbnent Psychology University " In a Team Robots Loudest Not . of The Is , , Agah A., andBekeyG. 1995a Necessarily theBest Proceedings theInternational on ," . Man of Conference Systems , andCybernetics , Vancouver , B.C. ." from Perception , , , Success Failure , and Agah A., andBekey G. 1995b Learning " in a Teamof Autonomous MobileRobots Proceedings the Seventh , of Portuguese on IslandPortugal . , (EPIA1995, Madeira ) Conference ArtificialIntelligence A., andBekeyG. 1997"Phylogenetic Onto . and in of , , Agah genetic Learning a Colony " Robots Autonomous RobotsVol. 1, No. 4, January . 85 100 . , , , pp Interacting ." : of of , . , D. Agre P E., andChapman 1987 Pengi An Implementation a Theory Activity " theAmerican Association ArtificialIntelligence , Proceedings of of Conference - ), -71. (AAAI 87 pp. 268 ." For " and , . , D. Agre P E. andChapman 1990 WhatAre Plans ? Robotics Autonomous Vol. 6, pp. 17 34. , Systems Albus J. 1981BrainsBehavio1 Robotics . . , ; and , BYTEBooksPeter , , NH. borough " . Albus J. 1991" Outline a Theory IntelligenceIEEETransactions Systems for of on , . , ManandCybernetics . 21, No. 3, May Junepp. 473 509 . , Vol , ." AlbusJ., McCainH. and / Standard Reference Modelfor , , Lumia R. 1987 NASA NBS , ArchitectureNASREM' NBSTechnical 1235Robot Note Telerobot Control , ( ): System DivisionNational Bureau Standards of . , Systems . Press Allee W., 1978Animal , , Univ , IL. Aggregations . of Chicago , Chicago Aloimonos (ed.) 1993Active . Erlbaum Assocciates , Lawrence , Hillsdale , Y. , Perception NJ. " AloimonosY., andRosenfeld 1991 " Computer . Vision Science . 253 pp. , , A. , , Vol , . 124953, September -

446

References

" . Altmann, S. A. 1974 " Baboons Space Time, and Energy AmericanZoologist Vol. , , , , 14 pp. 221 48. , Anderson J. A . 1995 "AssociativeNetworks" in TheHandbookof Brain Theoryand . , , Neural Networks ed. M. Arbib, MIT Press CambridgeMA , pp. 102 7. , , , "Animal Behavioras a . Anderson T., and Donath M. 1991 , , Paradigmfor Developing Robot Autonomy" in DesigningAutonomous , , , , Agents ed. P. Maes MIT Press Cambridge , MA , pp. 145 68. Andresen F., Davis, L., EastmanR., and KambhampatiS. 1985 " Visual Algorithms . , , , for AutonomousNavigation" Proceedings the IEEE International Conference on , of RoboticsandAutomation St. Louis, MO , pp. 856- 61. , Arbib, M.A. 1964 Brains, Machines and MathematicsMcGraw-Hill , New York. . , , Arbib, M . A. 1981 " Perceptual . Structures and DistributedMotor Control," in Handbook II of Physiology- TheNervousSystem : Motor Control, ed. V. B. Brooks, American , , PhysiologicalSociety BethesdaMD , pp. 1449 80. " Arbib, M . A. 1992 " Schema . , in The Encyclopediaof Artificial Intelligence , Theory 2nd ed., ed. S. Shapiro Wiley-interscienceNew York, N.Y., pp. 1427 43. , , " Arbib, M. A. 1995a " Schema . , Theory in TheHandbookof Brain Theoryand Neural Networks ed. M . Arbib , MIT Press Cambridge MA , pp. 830- 34. , , , Arbib, M. A. 1995b TheHandbookof Brain Theoryand Neural Networks ed. M. Ar . , bib, MIT Press Cambridge MA . , , Arbib , M ., andHouse D. 1987 " DepthandDetours An Essayon Visually GuidedBehavior . : , " , , , in Vision Brain, and Cooperative , Computationed. M. Arbib and A. Hanson MIT Press Cambridge MA , pp. 129 63. , , Arbib, M., Ibera , T., and Lyons D. 1985 " Coordinated Il . Control Programs Movements for , of the Hand" in Hand Function and the Neocortex eds A. Goodmanand . , , I. Darian Smith, SpringerVerlag New York, pp. 135 70. , . Arbib, M . A., Kfoury, A. J., and Moll , R. N. 1981 A Basisfor TheoreticalComputer ScienceSpringerVerlag New York. , , Arkin , R. C. 1986 " PathPlanningfor a Vision-BasedAutonomousRobot" Proceedings . , on , , of the SPIE Conference Mobile Robots Cambridge MA , pp. 240- 49. . Arkin , R. C. 1987a " Motor SchemaBasedNavigation for a Mobile Robot: An Approach on to Programming Behavior" Proceedings theIEEE Conference Robotics , of by and Automation Raleigh NC, pp. 264- 71. , , . Robots IntelligentNavigationin Extended : Arkin , R. C. 1987b"TowardsCosmopolitan " Man-Made Environments PhiD. Dissertation COINS TechnicalReport 87-80, University , , of . of Massachusetts , Department ComputerandInformationScience Arkin , R. C. 1988 " Homeostatic . Control for a Mobile Robot: DynamicReplanningin " on Hazardous Environments Proceedings the SPIE Conference Mobile RobotsIII , , of . 407- 13. , Cambridge MA , pp . in Arkin , R. C. 1989a " Neuroscience Motion: The Application of Schema Theory to : Mobile Robotics" in VisuomotorCoordination Amphibians ComparisonsModels , , , , and Robots eds J.-P. Ewert andM. Arbib, New York: PlenumPresspp. 649- 72. . , ,

447

References

Arkin , R. C. 1989b " Motor SchemaBasedMobile Robot Navigation" International . , Journal of RoboticsResearchVol. 8, No. 4, pp. 92- 112 . , Arkin , R. C. 1989c " NavigationalPath Planningfor a Vision-basedMobile Robot" . , Robotica Vol. 7, pp. 49- 63. , Arkin , R. C. 1989d "Towards the Unification of NavigationalPlanningand Reactive . Control," working notes AAAI SpringSymposium RobotNavigation StanfordUniversity on , , , CA , March. Arkin , R. C. 1990a "The Impact of Cyberneticson the Design of a Mobile Robot . " : on , , , System A CaseStudy IEEE Transactions SystemsMan, and Cybernetics Vol. 20, No. 6, November December . 1245 57. / , pp Arkin , R. C. 1990b " Integrating Behavioral Perceptual and World Knowledge in . , , " Reactive , , Navigation RoboticsandAutonomous SystemsVol. 6, pp. 105 22. " Arkin , R. C. 1991 " ReactiveControl as a Substrate TeleroboticSystems IEEE . for , and ElectronicsSystems Vol. 6, No. 6, June pp. 24- 31. , , Aerospace Magazine Arkin , R. C. 1992a " BehaviorBasedRobotNavigationfor Extended . Domains" Adaptive , Behavior Vol. 1, No. 2, pp. 201 225. , Arkin , R. C. 1992b " Cooperation . without CommunicationMultiagent Schema : Based RobotNavigation" Journal of RoboticSystemsVol. 9, No. 3, April , pp. 351 64. , , Arkin , R. C. 1992c " Homeostatic . Control for a Mobile Robot: Dynamic Replanning " in Hazardous Environments Journal of Robotic SystemsVol. 9, No. 2, March, pp. , , 197 214. Arkin , R. C. 1993 " Modeling Neural Function at the SchemaLevel: Implications . and Resultsfor Robotic Control," in Biological Neural Networksin InvertebrateNeu. , , , roethologyand Robotics eds T. McKenna R. Ritzmannand R. Beer SanDiego, CA, pp. 383 410. Arkin , R. C., and Ali , K. 1994 " Integrationof Reactiveand TeleroboticControl in . " on , MultiAgent Robotic Systems Proceedings the Third International Conference of Simulationof AdaptiveBehavior(SAB94 [ FromAnimals to Animats , Brighton, UK , ) ] , August pp. 473- 78. Arkin , R. C., Balch, T., Collins, T., HenshawA., MacKenzie D., Nitz, E., Rodriguez , , , R., and Ward K. 1993 " Buzz: An Instantiationof a SchemaBasedReactiveRobotic . , " on , System Proceedings the International Conference Intelligent Autonomous of Systems , , (IAS-3), Pittsburgh PA, Februarypp. 418- 27. Arkin , R. C., and Hobbs J. D. 1992 " Dimensionsof Communication . and SocialOr, " , ganizationin MultiAgent RoboticSystems FromAnimalsto Animats2: Proceedings International Conference Simulationof Adaptive~ ehavior, Honolulu, on of the Second HI , DecemberMIT Press Cambridge , pp. 486- 93. MA , , Arkin , R. C., andLawton, D. 1990 " Reactive . BehavioralSupportfor QualitativeVisual " on , Navigation Proceedings the IEEE International Symposium Intelligent Motion of Control, Istanbul Turkey 1990 pp. IP21- 28. , , , Arkin , R. C., and MacKenzie D. 1994 "Temporal Coordinationof Perceptual . , Algorithms for Mobile RobotNavigation" IEEE Transactions RoboticsandAutomation on , , Vol. 10 No. 3, June pp. 276- 86. , ,

448

References Arkin , R. C., and Murphy, R. R. 1990 "AutonomousNavigationin a Manufacturing . " Environment IEEE Transactions Roboticsand Automation Vol. 6, No. 4, August on , , , pp. 445- 54. Arkin , R. C., Murphy, R., PearsonM., and VaughnD . 1989 " Mobile Robot Docking . , " in in a ManufaCturing Environment Progress Visual Perceptual : , Strategies Operations on , Proceedings the IEEE International Workshop Intelligent Robotsand Systems of Tsukuba Japan pp. 147 54. , , . Aron, S., DeneubourgJ., Goss S., andPasteelsJ. 1990 " FunctionalSelf-Organization , , , lliustratedby Inter-NestTraffic in Ants: The Caseof the ArgentineAnt ," in Biological . Motion, eds W. Alt and G. Hoffmann SpringerVerlag Berlin, pp. 533- 47. , , ." , , Asada M., Noda S., TawaratsumidaS., and Hosoda K. 1995 Vision-BasedReinforcement , , " of Learning for PurposiveBehavior Acquisition, Proceedings the IEEE International Conference RoboticsandAutomation May, pp. 146 53. on , . , Ashby W. R. 1952 Designfor a Brain: The Origin of Adaptive Behavior, J. Wiley, New York 1952(Second edition 1960 . , ) " . AstrOm K. 1995 "AdaptiveControl: GeneralMethodology in TheHandbookof Brain , , , , , Theoryand Neural Networks ed. M. Arbib, Mit Press Cambridge MA , pp. 66- 69. " . Atkeson C., Moore, A ., andSchaal S. 1997 " Locally Weighted , , Learningfor Control, , , Artificial IntelligenceReview February Vol. 11. No. 1- 5, pp. 11 73. "A PracticalObstacle Detection . Badal S., Raveia S., Draper B., andHanson A. 1994 , , , , " IEEE Workshop Applicationsof on andAvoidanceSystem Proceedings the Second , of . , , , ComputerVision SarasotaFL , Decemberpp. 97- 104 " "Animation from Instructions in : . Badier N., and Webber B. 1991 , , , Making the Move . Mechanics Control and Animation of Articulated Figures eds Badier Barsky and , , , , Zeltzer MorganKaufmann SanMateo CA , pp. 51- 93. , , , " "Active . , , , of Perception Proceedings the IEEE, Vol. 76, No. 8, August Bajcsy R. 1988 . pp. 996- 1005 . Bakker P., and Kuniyoshi Y. 1996 " Robot See Robot Do: An Overview of Robot , , , on Imitation," AISB Workshop Learning in Robotsand Animals, Bright on, UK, April . : for . Balch, T., andArkin , R. C. 1993 "Avoiding thePast A Simplebut EffectiveStrategy on ReactiveNavigation" Proceedings the IEEE International Conference Robotics , of and Automation Atlanta GA , May, Vol. 1, pp. 678- 85. , , . Balch, T., and Arkin , R. C. 1994 "Communication in ReactiveMultiagent Robotic " Robots Vol. 1, No. 1, pp. 27- 52. , , Systems Autonomous . Balch, T., and Arkin , R. C. 1995 " Motor SchemaBasedFormationControl for Multiagent on 1995International Conference Multiagent Systems RobotTeams' Proceedings : , SanFrancisco CA , pp. 10 16. , Balch, T., Boone G., Collins, T., Forbes H., MacKenzie D., and Santamaria J. , , , , . 1995 " 10 Ganymedeand Callisto- A Multiagent Robot Trash Collecting Team AI ," , , , , Magazine Vol. 16, No. 2, Summerpp. 39- 51. Frames AnimateVision," Proceedings the Eleventh for ." of Ballard D. 1989 Reference , on International Joint Conference Artificial Intelligence(IJCAI-89), Detroit, MI , pp. 1635 41.

449

References " Ballard, D., and Brown, C. 1993 " Principlesof Active Perception in Active Perception . , , ed. Y. Aloimonos, LawrenceErlbaumAssociatesHillsdale, NJ, pp. 245 82. , Barber , A., Fitzgerald M., Albus, J., and Haynes L . 1984 " RCS: The NBS Real . a , , " time Control System Proceedings the Robots8 ConferenceDetroit, MI , June pp. , , , of 19.1 19.38. Barth, M ., and Ishiguro H. 1994 " Distributed PanoramicSensingin Multiagent . , Robotics" Proceedings the IEEE International Conference MultisensorFusion on , of and Integrationfor Intelligent SystemsLas VegasNY, October pp. 739- 46. , , , Bartlett, F. C. 1932 RememberingA Studyin Experimentaland Social Psychology . : , London, Cambridge . University Press " ." , , Basye K. 1992 An AutomataBasedApproachto Robotic Map Learning working notes AAAI Fall Symposium Applicationsof AI to Real-WorldAutonomous on Mobile , . Robots Beer R. 1990 Intelligence as Adaptive Behavior An Experimentin Computational . : , , , , NeuroethologyAcadelnicPress New York NY. Beer R., Chiel, H., and Sterling, L . 1990 "A Biological Perspective Autonomous . on , " , , Agent Design RoboticsandAutonomous SystemsVol. 6, pp. 169 86. " ." , , Bekey G., and Tomovic R. 1986 Robot Control by Reflex Actions, IEEE International on , , Conference Roboticsand Automation SanFrancisco CA , April , pp. 240- 47. " ." , G., andTomovic R. 1990 Biologically BasedRobotControl, Proceedings , Bekey of theAnnual International Conference the IEEE Engineeringin Medicineand Biology of , Society Vol. 12, No. 5, pp. 1938 39. Bellman R. 1978 Artificial Intelligence Can ComputersThink? Boyd and Frasier . : , , PublishingCo., Boston MA . Benson S., andNilsson, N. 1995 " Reacting Planning andLearningin anAutonomous . , , , " . , in MachineIntelligence14, eds K. Furukawa D. Michie, and S. Muggleton , , Agent Press Oxford, UK. Clarendon , Bersini, H. 1992 " Immune Network and Adaptive Control," Proceedings the First . of on , , EuropeanConference Artificial Ufe , Paris France pp. 217 26. Biedennan I. 1990 " Higher-Level Vision," in VisualCognitionand Action, Vol. 2, ed. . , N. OshersonS. Kosslyn andJ. Hollerbach MIT Press Cambridge MA . , , , , , Birnbaum L., Brand M ., and Cooper P. 1993 " Looking for Trouble: Using Causal . , , , Semantics Direct Focusof Attention," Proceedings the FourthInternational Conference to of on ComputerVision(ICCV-93), Berlin, Gennany May, pp. 49- 56. , -Ivaldi, F., and Giszter S. 1991 " Computations Bizzi, E., Mussa . , Underlying the Execution " of Movement A Biological Perspective ScienceVol. 253, July, pp. 287- 91. : , , " . Blake, A. 1993 " Computational , Modelling of Hand Eye Coordination in Active Perception ErlbaumAssocociatesHillsdale, NJ, pp. 227- 44. , ed. Y. Aloimonos Lawrence , , . Blake, A. 1995 "Active Vision," in The Handbookof Brain Theoryand Neural Networks , ed. M. Arbib , MIT Press Cambridge MA , pp. 61- 63. , , Boden M. 1995 "AI ' s Half-Century" AI Magazine Vol. 16, No. 2, Wmter, pp. 96- 99. . , , ,

450

References " ." of , , Bogoni, L., andBajcsy R. 1994a Active Investigation Functionality Proceedings on the Role of Functionality in Object Recognition 1994Conf. on , of the Workshop . , , ComputerVision andPatternRecognition Seattle WA, June ." , Using a DiscreteEvent Bogoni, L., and Bajcsy R. 1994b FunctionalityInvestigation " Roboticsand Autonomous , , , SystemsVol. 13, No. 3, October pp. SystemApproach 173 96. . Bohm, C., and Jacopini G. 1966 " Flow Diagrams Thring Machines and Languages , , , with Only 1WoFormationRules" Communications theACM, May. Vol. 9, No. 5, pp. , of 366- 71. BonassoP. 1991 " Underwater . , Experiments Using a Reactive Systemfor Autonomous " Vehicles Proceedings theAAAI , pp. 794- 800. , of " Bonasso P. 1992 " ReactiveControl of UnderwaterVehicles Applied Intelligence . , , , Vol. 2, No. 3, Septemberpp. 201- 04. , . and Booker L., Goldberg D., andHolland J. 1989 " ClassifierSystems GeneticAlgorithms , , , " , Artificial Intelligence Vol. 40, No. 1-3, pp. 235 82. , . Borenstein J., and Koren Y. 1989 " Real Time ObstacleAvoidancefor Fast Mobile , , Robots" IEEE Transactions Systems , and CyberneticsVol. 19, No. 5, September on , Man , , , pp. 1179 87. . VectorField Histogram FastObstacle Avoidance BorensteinJ., andKoren Y. 1991 "' The , , and for Mobile Robots" IEEE Transactions Robotics Automation Vol. 7, No. 3, on , , June pp. 278- 88. , " : . Bower T. 1974 "' TheEvolution of SensorySystems in Perception Essaysin Honor , , . J. , , , , of James Gibson eds R. MacLeodandH. Pick Cornell UniversityPressIthaca NY, . p. 141 : . Box, H. 1973 Organisationin Animal Communities Experimentaland Naturalistic Studies the SocialBehaviorof Animals, Butterworths London. , of W. 1971 An Introductionto the Studyof Disease Lea & Febiger Philadelphia . . , , Boyd, PA. " . " , , Brady M . 1985 Artificial Intelligence and Robotics Artificial Intelligence and . Robotics Vol. 26, pp. 79- 121 , : in . , , , PsychologyMIT PressCambridge BraitenbergV. 1984 VehiclesExperiments Synthetic , MA . " . and Brill , F. 1994 " Perception Action in a Dynamic Three DimensionalWorld, Proceedings on , , , of the IEEE Workshop VisualBehaviors Seattle WA, June pp. 60- 67. " . Brooks R. 1986 "A Robust Layered Control System for a Mobile Robot IEEE , , -2, No. I , pp. 14 23. Journal of RoboticsandAutomation Vol. RA , " . Brooks R. 1987a " Planningis Justa Wayof Avoiding Figuring Out What to Do Next, , . , WorkingPaper303, MIT AI Laboratory September "A Hardware Distributed Layered Architecture for . Brooks R. 1987b , Retargetable on Mobile RobotControl," Proceedings theIEEE InternationalConference Robotics of and Automation Raleigh NC, May, pp. 106 10. , ,

451

References Brooks R. 1989a"A RobotThat Walks: Emergent . Behaviors from a CarefullyEvolved , " on Network Proceedings the IEEE International Conference RoboticsandAutomation , of , May, pp. 692 94. Brooks, R. 1989b "The Whole Iguana" in RoboticsScienceed. M . Brady MIT Press . , , , , , Cambridge MA , pp. 432- 56. " Brooks R. 1990a "The BehaviorLanguage A.I. MemoNo. 1227, MIT AI Laboratory . , , , April . " Brooks R. 1990b " Elephants ' t Play Chess in DesigningAutonomous . Don , , , Agents ed. P. Maes MIT Press Cambridge MA , pp. 3- 15. , , , " Brooks R. 1991a " IntelligenceWithout Reason A.I. MemoNo. 1293 MIT AI Laboratory . , , , , April . Brooks R. 1991b " New Approach to Robotics" Science Vol. 253, Septemberpp. . es , , , , 1227 32. . Brooks R., and Flynn, A. 1989 " Robot Beings" Proceedings the IEEE RSJInternational . , / , of on Conference Intelligent Roboticsand SystemsIROS 89), Tsukuba Japan ( , , pp. 2- 10. Brooks R. A., and Stein L . 1994 " Building Brains for Bodies" Autonomous . , Robots , , , Vol. 1, No. 1, pp. 7- 25. Brown, C. 1991 " GazeBehaviorsfor Robots" in Active Perception Robot Vision . and , , eds A. SoodandH. WechslerSpringerVerlag Berlin, pp. 115 39. . , , Budenske J., and Gini, M . 1994 " Why Is It So Difficult for a Robot to Passthrough . , a DoorwayUsing illtrasonic Sensors" Proceedings the IEEE International Conference ? of on Roboticsand Automation pp. 3124 29. , Buhler, M., Koditschek D., and Kindlmann P. 1989 "A Family of Robot Control . , , " for , Strategies IntemrittentDynamicalEnvironments Proceedings the International of on . , , Conference Roboticsand Automation ScottsdaleAZ , pp. 1296 1301 ." Information , M., andMittelstaedt H. 1972 CourseControl by StoredProprioceptive , Burger in Millipedes," BiocyberneticsVol. IV, ed. H. DrischelandP. Dettmar Fischer , , . , Verlag Berlin, June 1972 . , , , , , , Byrnes R., Healey A., McGhee R., Nelson R., Kwak S., and Brotzman D. 1996 "The RationalBehaviorSoftwareArchitecturefor " , Intelligent Ships Naval Engineers Journal, Vol. 108 No. 2, March, pp. 43- 55. , Cal, A ., Fukuda T., Araj, F., Ueyama T., andSakal A. 1995 " HierarchicalControl Architecture . , , , " for Cellular RoboticSystem SimulationsandExperiments Proceedings , of the IEEE International Conference RoboticsandAutomation June pp. 1191 96. on , , CameronJ., MacKenzie D., Ward K., Arkin , R., andBook, W. 1993 " ReactiveControl . , , , for Mobile Manipulation" Processing the International Conference Robotics on , of and Automation Atlanta GA, pp. 228 --35. , , Cao Y., FukunagaA., Kabog A., andMeng, F. 1995 " Cooperative . Mobile Robotics : , , , Antecedents Directions" Proceedings the IEEE RSJInternational Conference and / , of on Intelligent Roboticsand SystemsIROS ' 95), Pittsburgh PA, pp. 226 --34. , (

452

References . of Cardoze D., and Arkin , R. C. 1995 " Development Visual TrackingAlgorithms for , " X an Autonomous , , Helicopter Procee ~ingsof theMobile Robots , PhiladelphiaPA, pp. 145 56. and MathematicalOrganizationTheory: Perspective ." , Carley K. 1995 Computational " and Directions Computationand MathematicalOrganizationTheory Vol. 1, No. 1, , , pp. 39- 56. : . Carr, G. M ., and MacDonaldD . 1986 "The Sociality of Solitary Foragers A Model " Basedon Resource , Dispersion Animal Behavior, Vol. 34, pp. 1540 49. " -Perez F. 1995 " VisuomotorCoordination FrogsandToads in TheHandbook in . Cervantes , , , , , of Brain Theoryand Neural Networks ed. M. Arbib, MIT Press CambridgeMA , pp. 1036 42. " . Chalmers D. 1995 "The Puzzleof Conscious , , , Experience ScientificAmerican Vol. . 273, No. 6, pp. 80- 86, December " ." , , , Chapman D. 1990 lntennediateVision: Architecture Implementation and Use, . TechnicalReportTR-90 06, TeleosResearchPaloAlto , CA, October , . L., and Lob, J. 1994 "Coordination and Control of a Group of Small Robots" Chen , , on , of Proceedings the International Conference Roboticsand Automation May, pp. 2315 20. : . ChristensenH., Bowyer, K., and Bunke, Heds . 1993 Active Robot Vision Camera , . Heads Model-BasedNavigationand ReactiveControl, World Scientific Singapore , , " . ChristiansenA., Mason M., and Mitchell , T. 1991 LearningReliableManipulation , , without Initial PhysicalModels ' RoboticsandAutonomous , : SystemsVol. 8, Strategies No. 1, pp. 7- 18. . GroundVehicleDemo II : Demonstration Chon, W., and Jochem T. 1994 " Unmanned , Wmter, pp. 14 20. A ," Unmanned , Systems ." Chun, W., Lynch, R., ShoemakerC., and Munkeby S. 1995 UGV- Demonstration , , " Unmanned . 20- 25. B, , , SystemsSummer pp : ." , Clark R. J., Arkin , R. C., and Ram A . 1992 Learning Momentum On-Line Performance , " for Enhancement ReactiveSystems Proceedings the IEEE International , of on , , Conference RoboticsandAutomation Nice, France May, pp. 111 16. SocialRecognition J. Wiley, New York. . , , Colgan P. 1983 Comparative . T. R., Arkin , R. C., and HenshawA. M . 1993 " Integrationof ReactiveNavigation Collins, , " with a Flexible ParallelHardwareArchitecture Proceedings the IEEE International , of on , , Conference Roboticsand Automation Atlanta GA, May, Vol. 1, pp. 271 76. . Colombetti M., and Dorigo, M. 1992 " Learning to Control an AutonomousRobot , " of by DistributedGeneticAlgorithms, From Animals to Animats 2: Proceedings the on SecondInternational Conference Simulationof Adaptive Behavior, Honolulu, HI , MA DecemberMIT Press Cambridge , pp. 305 12. , , " Creature Architecture" Proceedings . Connell J. 1987 , , Building with the Subsumption on Artificial Intelligence(IJCAI-87), Milan , Italy, of the InternationalJoint Conference pp. 1124 26.

453

References Connell J. 1989a"A BehaviorBased . Arm Controller" lE EE Transactions Robotics on , , and Automation Vol. 5, No. 6, Decemberpp. 784- 91. , , " Connell J. 1989b "A Colony Architecturefor an Artificial Creature Technical . , , Report No. 1151 MIT AI Laboratory August. , , Connell, J. 1992 " SSS A Hybrid ArchitectureApplied to RobotNavigation" Proceedings . : , the IEEE International Conference Roboticsand Automation Nice, France on , of , --24. pp. 2719 Connell J., and Viola P. 1990 " Cooperative . Control of a Semi AutonomousMobile , Robot" Proceedings the IEEE International Conference Roboticsand Automation on , of , pp. III8 --21. ." , , Connolly C., and Gropen R. 1993 On the Applications of Harmonic Functionsto " Journal Robotic Robotics , , of SystemsVol. 10, No. 7, pp. 931- 46. D., Gmytrasiewicz P., and Holder, L. 1996 " DecisionTheoreticCooperative . Cook , , ,SensorPlanning" IEEE Transactions PatternAnalysis and Machine Intelligence on , , Vol. 18 No. 10, October pp. 1013 23. , , . : Craig, J. 1989 Introduction to Robotics Mechanicsand Control, 2nd Ed., Addison , Reading MA . , Wesley " Crisman J. 1991 " Color RegionTracking for VehicleGuidance Active Vision MIT . , , , Press Cambridge MA , pp. 107 20. , , ." , , , , Crowley J., Bedrune J., Bekker M ., andSchneiderM . 1994 IntegrationandControl " of Visual Process , Proceedings the IEEE Workshop VisualBehaviors Seattle es on , , of WA, June pp. 45- 52. , ." , , , , Croy, M ., and Hughes R. 1991 Effects of Food Supply Hunger Danger and Competition " on Choice of ForagingLocation by the Fifteen spinedStickleback Animal , Behavior Vol. 42, pp. 131 39. , Culhane S., and Tsotsos J. 1992 "An Attentional Prototypefor Early Vision," Proceedings . , , on , , of the SecondEuropeanConference Computer Vision ed. G. Sandini LNCS-SeriesVol. 588, SpringerVerlag Berlin, May, pp. 551- 60. , Davis, R. 1982 "Applicationsof MetaLevel Knowledgeto the Construction Maintenance . , " in , and Useof Large KnowledgeBases in KnowledgeBasedSystems Artificial , . , , , Intelligence eds R. Davis andD. Lenat McGraw-Hill , New York pp. 229- 490. Dean T., Angluin, D., Basye K., Engelson S., Kaelbling, L., Kokkevis E., and , , , , Maron, O. 1995 " Inferring Finite Automatawith Stochastic . Output Functionsand an " . , , Application to Map Learning MachineLearning, Vol. 18, No. I , pp. 81- 108 Jan -Kaufmann San Dean T., and Wellman M. 1991 Planning. and Control, Morgan . , , , Mateo CA. , " ." and , , DeneubourgJ., and Goss S. 1984 CollectivePatterns Decision Making, Ethol- 311. , , , ogy Ecology and Evolution Vol. 1, pp. 295 " ." Dennett D. 1982 Stylesof Mental Representation Proceedings the Aristotelian , , of - 16. , Society Vol. LXXXIII , pp. 213 . Dennett D. 1991 Consciousness , , , Explained Little , Brown, andCo., Boston MA .

454

References

Dickmanns E. 1992 "A GeneralDynamic Vision Architecturefor UGV and UAV," . , , , Applied Intelligence Vol. 2, No. 3, Septemberpp. 251- 70. Dickmanns E., andZapp A. 1985 " Guiding Land VehiclesalongRoadways Computer . , , by Vision," AFCET ConferenceToulouse France . , , Donald B. 1993 " Information Invariantsin Robotics Partll - SensorsandComputation . : , " on , Proceedings the International Conference RoboticsandAutomation Vol. 3, , of pp. 284- 90. Donald B., andJenningsJ. 1991a " Sensor . and , , Interpretation Task DirectedPlanning " Classes Proceedings the International Conference , Using Perceptual Equivalence of on Roboticsand Automation Anahelm CA, pp. 190 97. , , Donald B., andJenningsJ. 1991 " Perceptual b. Limits, Perceptual Classes , , , Equivalence " and a Robot' s Sensor-Computational i / , Proceedings the IEEE RSJInternational Capabilities of ' on . ( Conference IntelligentRoboticsand SystemsIROS 91), pp. 1397 1405 "Constructive Donald B., JenningsJ., and Brown, R. 1992 . for Task , , Recognizability " DirectedRobotProgramming RoboticsandAutonomous , , SystemsVol. 9, No. 1 2, pp. 41- 74. . : Drexler E. 1986 Enginesof Creation The ComingEra of NanotechnologyAnchor , , Press Double , New York. / day Drexler E. 1992 NanosystemsMolecular Machinery Manufacturingand Computation . : , , , Wiley-interscienceNew York. , Duchon A., WarrenW., and Kaelbling, L. 1995 " Ecological Robotics Controlling . : A Behaviorwith Optic Flow," Proceedings the SeventeenthMual Conference the of of , Society pp. 164 69. CognitiveScience ." Dudek G., Jenkin M ., Milios , E., and Wilkes, D. 1993 A Taxonomyfor Swarm , , Robots" Proceed ?s C the IEEE RSJ in , ? / International Conference Intelli ?ent Robots on , f ,

' andSystems , Japan . 441 47. , pp (IROS93 Yokohama ), " "A Sonar -Based BIfes A. 1986 . and , , of Mapping Navigation System Proceedings the IEEEInternational and San on Robotics Automation Francisco , pp. , , CA Conference 115156. Think " AI Magazine . 13 No. 2, Summer . '! ." , Vol , , pp , EpsteinR. 1992 CanMachines 80 95. -Roth . II Ennan L., Hayes , F., LesserV., andReddyD. 1980"The Hearsay Speech , , , " : to , Uncertainty Computing Understanding SystemIntegrating Knowledge Resolve -53. , Vol , Surveys . 12 No. 2, pp. 213 -Inspired ." Hexapod , , , K. , Espenscheid , QuinnR., Chiel H., andBeerR. 1994 Biologically " on and Robot Control ProceedingstheFifthInternational , of Conference Robotics ' - . , , (ISRAM94 AugustMaul HI, pp. 89 102 ), Manufacturing EverettB. 1995Sensorsfor . MobileRobots , MA. , A.K. Peters , Wellesley , to Fundamentals Ewert J-P 1980Neuroethology Introduction theNeurophysiological . . : An , -VerlagBerlin . , , of BehaviorSpringer Erlbaum Associates . , , M. , Lawrence Psychology of Eysenck 1993Principles Cognitive HoveUK. ,

455

References

-Learning Approach ." , , , Fagg A., Lotspeich D., and Bekey G. 1994 A Reinforcement Robots" Proceedings the IEEE to ReactiveControl Policy Designfor Autonomous , of on International Conference Roboticsand Automation pp. 39- 44. , Ferrell, C. 1994 " RobustAgent Control of an AutonomousRobot with Many Sensors . andActuators" MiS. Thesis MIT AI Laboratory CambridgeMA . , , , , "The HarvardBinocularHead" in Active Robot Vision Ferrier N., andClark J. 1993 . : , , , . CameraHeads Model-BasedNavigation and ReactiveControl, eds H. Christensen , , K. Bowyer andH. Bunke World Scientific Singaporepp. 9- 31. , , , , . : Fikes R., and Nilsson, N. 1971 " STRIPS A New Approachto the Application of , TheoremProvingto ProblemSolving," Artificial Intelligence Vol. 2, pp. 189 208. , " PhiD. Dissertation " . , Firby, R. J. 1989 AdaptiveExecutionin ComplexDynamic Worlds / , , TechnicalReportYALEU/CSD RR#672, YaleUniversity New Haven CT. , ." Learnedfrom the Animate Agent Project(So Far)," working Firby, R. J. 1995 Lessons on Learnedfrom Implemented notes AAAl Spring Symposium Lessons , Software Architectures PhysicalAgents PaloAlto , CA, March, pp. 92- 96. , for ." : , Firby, R. J., and Slack M. 1995 Task Execution Interfacingto ReactiveSkill Networks " on Learned , , working notes AAAl SpringSymposium Lessons from Implemented . Architectures PhysicalAgents PaloAlto , CA, March, pp. 97- 111 , for Software . : Fite, K. 1976 TheAmphibianVisualSystemA Multidisciplinary Approach Academic , Press New York. , . Floreano D., and Mondada F. 1996 " Evolution of Homing Navigation in a Real , , on Mobile Robot" IEEE Transactions SystemsMan and CyberneticsVol. 26, No. 3, , , , , June pp. 396- 407. , . : FlorenceS ., and Kaas J. 1995 " Somatotopy Plasticity of SensoryMaps" in The , , Handbookof Brain Theoryand Neural Networks ed. M . Arbib, MIT Press Cambridge , , MA , pp. 888- 91. Robotics " Working Paper ." ), Flynn, A. 1987 GnatRobots(andHow They Will Change . No. 295, MIT AI Laboratory Cambridge MA , June , , " ." , , , Flynn, A., and Brooks R. 1989 Battling Reality AI Memo No. 1148 MIT AI . , , Laboratory Cambridge MA , October :A ." , Flynn, A., Brooks R., andTavrowL. 1989 1Wilight ZonesandCornerstones Gnat " A.I. Memo No. 1126 MIT AI Robot Double Feature , Cambridge MA , , , , Laboratory July. ." , Flynn, A., Brooks, R., Wells, W., and Barrett D. 1989 Squirt: The Prototypical " A.I. Memo No. 1120 MIT AI Mobile Robot for AutonomousGraduateStudents , , , Laboratory CambridgeMA , July. ." , Fok K.-Y., and Kabuka MR . 1991 An Automatic Navigation Systemfor Vision , " GuidedVehiclesUsing a DoubleHeuristicanda Finite StateMachine IEEE Transactions , on Roboticsand Automation Vol. 7, No. 1, February pp. 181 88. , , " . FranceshiniN., Pichon J., andBianes C. 1992 " From InsectVision to RobotVision, , , , ThePhilosophicalTransactions theRoyalSocietyof LondonB, Vol. 337, pp. 283- 94. of

456

References Franceshini N., Riehle, A., and Le Nestour A. 1989 " Directionally SelectiveMotion . , , " Detectionby InsectNeurons in Facetsof Vision eds Slavenga Hardie Springer . and , , , , Verlag Berlin, pp. 360- 90. Franklin, R. F., and Hannon L . A. 1987 Elementsof Cooperative . Behavior Internal . , Research and Development Final Report 655404 1-F , EnvironmentalResearch Institute of Michigan ( BRIM), Ann Arbor, MI . FranklinD ., Kahng A., and Lewis, M . 1995 " DistributedSensingand Probing with . , Search : TowardSystemLevel LandmineDetectionSolution," in Proceedings Multiple Agents DetectionTechnologies Mines and Minelike Targets SPIE Vol. 2496 pp. , for , 69&- 709. Franks N. 1986 "TeaIns in Social Insects Group Retrievalof Prey by Anny Ants," . : , BehavioralEcologyand Sociobiology Vol. 18, pp. 425- 29. , Fu, D., Hammond K., and Swain M. 1994 " Vision and Navigation in Man-made . , " Environments Looking for Syrup in All the Right Places Proceedings the IEEE : , of on , , , Workshop VisualBehaviors Seattle WA, June pp. 20- 26. . Fukuda T., Nakagawa S., Kawauchi Y., and Buss M . 1989 " StructureDecision , , , , for Self OrganisingRobots Basedon Cell Structures CEB( Jf," IEEE International on , , Conference Roboticsand Automation ScottsdaleAZ , pp. 695 70. " CommunicationReductionwith Risk . Fukuda T., and Sekiyama K. 1994 , Estimate , " for Multiple Robotic System Proceedings the IEEE International Conference on , of Roboticsand Automation pp. 2864 69. , . Gachet D., Salichs M ., Moreno L., andPi mentalJ. 1994 " LearningEmergent , , Tasks , , for an AutonomousMobile Robot" Proceedings the International Conference on , of ' ( Intelligent Robotsand SystemsIROS 94), Munich, GermanySeptemberpp. 290- 97. , , " ." , Gage D. 1992 SensorAbstractionsto SupportMany-Robot Systems Proceedings , --46 , , of Mobile RobotsVII , Boston MA , Novemberpp. 235 . J., and Beer R. 1992 "A QualitativeDyna . Inical Analysisof EvolvedLocomotion , , Gallagher Controllers" From Animals to Animats2: Proceedings the Second International , of on Conference Simulationof AdaptiveBehavior, Honolulu, In , DecemberMIT , Press Cambridge , pp. 71- 80. MA , Gallistel, C. R. 1980 TheOrganizationof Action: A NewSynthesisLawrenceErlbaum . , AssociatesHinsdale NJ. , , Gallistel, C. R. 1990 The Organizationof Learning, MIT Press Cambridge MA . . , , Gardner H. 1985 The Mind 's New Science A History of the CognitiveRevolution . : , , BasicBooks New York. , . and , , Garey M ., andJohnsonD. 1979 Computers Intractability: A Guideto the Theory and , W.H. Freeman Co., SanFrancisco CA. , of NP-Completeness Gat, E. 1991a " Reliable Goal-Directed Reactive Control of Autonomous Mobile . Robots" Ph. D Dissertation Virginia Polytechnic Institute and State University . , , , Blacks burg.

Gat E. 1991 "ALFA: A Language Programming b. for Reactive Robotic ControlSystems , " on and , Proceedings theIEEEInternational , of Conference Robotics Automation Sacramento , pp. 111620. , CA

457

References

in ." and Gat E. 1992 Integrating , Asynchronous Planning Reaction a Heterogeneous " for Real Architecture Controlling -WorldMobileRobots Proceedings theAAAl . , of Mobile Gat E. 1995 "TowardsPrincipledExperimental , , Study of Autonomous " Robots Autonomous RobotsVol. 2, No. 3, pp. 17989. , , ~ Gat E., andDoraisG. 1994" Robot ," Proceedings , , Sequencing Navigation Conditional by onRobotics Automation . 129399. and theIEEEInternational , pp Conference of "A -LineLearning : Neural for On . Gaussier ., andZrehenS. 1994 Topological , ,P " Map in Avoidance a MobileRobot Proceedings theThirdConference of , of Emergence Obstacle to 3 BehaviorFrom Animals Animats ), MIT Press on Simulation Adaptive , ( of , MA, pp. 282 90. Cambridge Artificial . Ac: Architecture Control to Gaussier., andZrehenS. 1995" Per A Neural ,P , . and Animals Robotics Autonomous , Vol , ," Systems . 16 pp. 291 320 " and ." , , M. of Reasoning PlanningProceedings Georgeff andLanskyA. 1987 Reactive .theAAAl -87, pp. 677 82. in ." and , , M. , GeorgeffM., LanskyA., andSchoppers 1986 Reasoning Planning Dynamic " Note . : with DomainsAn Experiment a MobileRobot SRITechnical No 380 , , . AI CenterSRIInternational , " ." , Vol , , A. of Georgopoulos 1986 On ReachingAnnualReview Neuroscience . 9, pp. - 70. 147 -Hall to . : An GeraldM. C. 1981Pharmacology Introduction Drugs Prentice , Engle , , woodCliffs, NJ. Brain" in TheBrain W.H. Freeman . of Geschwind 1979" SpecializationstheHuman , , , N. - . , NewYork pp. 10817 , Mifflin, to . GibsonJ. J. 1979TheEcological , Houghton , Perception Approach Visual BostonMA. , Bad ," : An ." ) , M. Planning (Almost Universally Idea A/ Magazine Ginsberg 1989 Universal , , Vol. 10 No. 4, Winterpp. 40 44. , and . Giralt G., Chatila R., andVaissetM. 1984"An Integrated , , , Navigation Motion " Mobile Robots First International for ControlSystem Autonomous , Multisensory . Research . M. BradyandR. Paulpp. 191 214 on , , ed Symposium Robotics . de an Goel A., Ali , K., Donnell , M., Gomez SilvaGarzaA., andCallantine 1994 , T. , , " PathPlanning IEEEExpert Vol. 9, No. 6, December . , pp , ," Multistrategy Adaptive 57 65. Press of . Ants Goetsch 1957The . University Michigan , , AnnArbor MI. , W. Learning and in SearchOptimization Machine . D. 1989Genetic , , , Algorithms Goldberg -Wesley MA. Addison , Reading , Controlof an Fusion Reactive for ." , , S. Goodridge andLuo R. 1994 FuzzyBehavior " : Autonomous MobileRobotMARGE Proceedings theIEEEInternational , Conference of and onRobotics Automation . 162227. , pp . Goss S., BeckersR., Deneubourg , Arnn S., andPasteels 1990 " How Trail , J. , , J. , , Mechanisms Problemsin Behavioral Can andTrailFollowing Solve ," Foraging Laying -VerlagHeidelberg Food Selection . R. Hughes , Germany . 661 78. , pp , , Springer , ed of

458

References Graefe V. 1990 "An Approach to ObstacleRecognition for AutonomousMobile . , Robots" Proceedings the IEEE International Workshop Intelligent Robotsand on , of ' ( SystemsIROS 90), pp. 151 58. Grefenstette J. and Schultz A. 1994 "An Evolutionary Approach to Learning in . , , Robots" MachineLearning Workshop Robot Learning, New Brunswick, NJ, July. on , Also availableas NCARAl ReportAlC -94-014, Navy Centerfor Applied Research in Artificial Intelligence WashingtonDC. , , ." Behaviors- MappingPerception , , Grupen R., andHendersonT. 1990 Autochthonous " in Traditional and Non-Traditional to Action, Robotic Sensors ed. T. Henderson , , NATO ASI Series Vol. F-63, SpringerVerlag Berlin, pp. 285 311. , , " ." , , , Guigon E., and Burnod Y. 1995 Short Term Memory in The Handbookof Brain , Theoryand Neural Networks ed. M . Arbib, MIT Press Cambridge MA , pp. 867 71. , , "1991 " . Experiments with the Subsumption Architecture" , , Hartley R., and Pipitone F. , on Proceedings theIEEE InternationalConference RoboticsandAutomation Sacramento of , , CA , pp. 1652 8. " ." , Hayes G., and Demiris, J. 1994 A Robot Controller Using Learning by Imitation, on Proceedings the SecondInternational Symposium Intelligent Robotic Systems of , Grenoble France pp. 198 204. , , " ." HayesRoth, B. 1995 An Architecture for Adaptive Intelligent Systems Artificial , - 2, Januarypp. 329- 65. , , Intelligence Vol. 72, No. 1 -Roth, B., Lalanda P., Morignot, P., Pfleger K., andBalabanovicP. 1993 " Plans . , , , Hayes andBehaviorin IntelligentAgents" Technical , ReportKSL- 93-43, KnowledgeSystems , , , Laboratory StanfordUniversity Stanford CA. -Roth, B., Pfleger K., Morignot, P., Lalanda P., and Balabanovic M. 1995 . , , , Hayes "A Domain " -Specific SoftwareArchitecturefor Adaptive Intelligent Systems IEEE , Transactions SoftwareEngineering Vol. 21, No. 4, April , pp. 288- 301. on , " ." in , M ., Ballard, D., and Pelz J. 1994 Visual Representations NaturalTasks , , Hayhoe on , , Proceedings the IEEE Workshop VisualBehaviors Seattle WA, June pp. 1- 9. of , Head H. andHolmes G., 1911 " Sensory . Disturbances from CerebralLesions" Brain, , , , Vol. 34, p. 102 . Hebb D. 1949 TheOrganizationof Behavior, New York Wiley. . , , Hexmoor H. and Kortenkamp D. 1995 " Issueson Building Softwarefor Hardware . , , " , , Agents Knowledge EngineeringReview Vol. 10, No. 3, pp. 301 04. Hexmoor H., Kortenkamp D., Arkin , R., Bonasso P., and Musliner Deds . 1995 . , , , , Learned Architectures Physical , Working Notes Lessons from Implemented Software for Series March, Stanford CA. , , , Agents AAAI Spring Symposium Hillis , D. 1988 " Intelligence as EmergentBehavior or the Songsof Eden" in The . , , : , , , , Artificial IntelligenceDebate FalseStarts Real FoundationsMIT Press Cambridge MA . Hinton, G., and Sejnowski T. 1986 " Learning and Relearningin Boltzmann Machines . , " . , in Parallel DistributedProcessingVol. 1, eds D. RumelhartandJ. McClellan, , MIT Press Cambridge MA , pp. 282 317. , ,

459

References ." : , , Hodgins J., and Brogan D. 1994 Robot Herds Group Behaviorsfor Systemswith " , Artificial Ufe IV , MIT Press Cambridge MA , pp. 319- 24. , , SignificantDynamics " " . Creatures MIT Epistemol , , Hogg, D., Martin, F., and Resnick R. 1991 Braitenberg andLearningMemorandum No. 13, Cambridge MA . , ogy " ." Transitionsin CombinatorialSearch , Hogg, T. 1996 QuantumComputingandPhase - 128 Journal of Artificial IntelligenceResearchVol. 4, pp. 91 . , Holldobler B., andWilson, E. 1990 TheAnts, BelknapPress CambridgeMA . . , , , " " PhiD. Dissertation Department Horswill, I. 1993a Specialization Perceptual . of Process , es , of Electrical Engineeringand ComputerScience Massachusetts Institute of , , , TechnologyCambridge MA May. Horswill, I. 1993b " Polly, A Vision-BasedArtificial Agent" Proceedings theAAAl . , of 93, WashingtonDC, pp. 824- 29. , Horswill, I., and Brooks R. A . 1988 " SituatedVision in a Dynamic Environment . : , . " National Conference Artificial Intelligence on , of ChasingObjects Proceedings the Seventh ' , , (AAAl 88), St. Paul MN , August pp. 796- 800. "A $1000Active StereoVision " Horswill, I., and Yamamoto M . 1994 . , , System Proceedings the IEEE Workshop VisualBehaviors Seattle WA, June pp. 107 10. on , , , of " ." , Huang H-M. 1996 An Architectureanda Methodologyfor IntelligentControl, IEEE and , Expert: Intelligent Systems Their Applications Vol. II , No. 2, April , pp. 46- 55. . Huber M., and Durfee E. 1995 " Deciding When to Commit to Action During , , " -BasedCoordination Proceedings the First International Conference Observation , of '95 SanFrancisco CA on MultiagentSystemsICMAS ), , , pp. 163 69. ( : Hull , C. 1943 Principlesof Behavior An Introductionto BehaviorTheory Appleton . , Century Crofts, New York. ." Learning , fig , W., andBerns K. 1995 A LearningArchitectureBasedon Reinforcement for Adaptive Control of the Walking Machine LAURON," Roboticsand Autonomous , , SystemsVol. 15, October pp. 321 34. ." , , , Ishiguro A., Ichikawa S., and Uchikawa Y. 1994 A Gait Acquisition of a Six" , of LeggedRobotUsing ImmuneNetworks Proceedings theInternational Conference on Intelligent Roboticsand SystemsIROS ' 94), Munich, Germanypp. 1034 41. , ( ." , H., Maeda T., Miyashita T., and Tsuji, S. 1994 A Strategyfor Acquiring , , Ishiguro " an Environmental Model with Panoramic , of Sensing a Mobile Robot Proceedings by on the IEEE International Conference Roboticsand Automation SanDiego, CA , pp. , 724- 29. " . Jablonski J., and Posey J. 1985 " RoboticsTerminology in Handbookof Industrial , , , . 1271 1303 . Robotics ed. S. Nof, J. Wiley, New York pp , , ." , Janet J., Schudel D., White, M ., England A ., and Snyder W. 1996 Global Self, , , Knowledge Localizationfor Actual Mobile Robots Generating SharingTopographical : and " , of Using the RegionFeatureNeural Network Proceedings the IEEE International . on , , Conference MultisensorFusionand Integration WashingtonDC, December " "The Immune Jeme N. 1973 . , , , System ScientificAmerican Vol. 229, pp. 52- 60.

460

References Johnson P., and Bay, J. 1995 " DistributedControl of SimulatedAutonomousMobile . , " Robot Collectivesin PayloadTransportation Autonomous Robots Vol. 2, No. I , pp. , , 43- 64. Johnson S. D. 1983 Synthesis Digital Designsfrom RecursionEquations MIT . , , of Press Cambridge MA . ' , , Kaas J., Krubitzer L., Chino, Y., Langston A., Polley E., andBlair, N. 1990 " Reorganization . , , , , of RetinotopicCortical Maps in Adult Mammalsafter Lesionsof the Retina" , Science Vol. 248, April , pp. 229 31. , " ." , Kaelbling, L. 1986 An Architecturefor Intelligent ReactiveSystems SRI International Technical Note No. 400, Menlo Park CA , October . , ." for Kaelbling, L. 1987 REX: A Symbolic Language the Design and Parallel Implementation " of Embedded on , Systems Proceedings the AIAA Conference Computers of in Aerospace , Wakefield MA , pp. 255 60. VI , " ." , S. Kaelbling, L., and Rosenschein 1991 Action and Planningin Embedded , Agents in DesigningAutonomous ed. P. Maes MIT Press CambridgeMA , pp. 35- 48. , , , , Agents " Kahn, P. 1991 " Specification Control of BehavioralRobotPrograms Proceedings . and , . , of the SPIESensorFusionIV , Boston MA , November Kaufmann J. 1974 " Social Ethology of the Whiptail Wallaby MacropusParryi, in . , , Northeastern New SouthWales" Animal Behavior, Vol. 22, pp. 281 369. , " Multilevel Path . , , , Keirsey D., Mitchell, J., Payton D., and Preyss E. 1984 Planning " for AutonomousVehicles SPIE Vol. 485, Applicationsof Artificial Intelligence pp. , , 133 37. ." for Kelly, M . and Levine, M . 1995 WhereandWhat: Object Perception Autonomous " Robots Proceedings the IEEE International Conference Roboticsand Automation on , of , pp. 261 67. , , , , Kenny P., Bidlack, C., Kluge, K., Lee, J., Huber M., Durfee E., and Weymouth T. . 1994 " Implementationof a ReactiveAutonomousNavigationSystemon an Outdoor Mobile Robot" Proceedings the Associationof Unmanned VehicleSystems Annual , of , SymposiumDetroit, MI , May, pp. 233 39. Khatib, O. 1985 " Real Time Obstacle . Avoidance Manipulators Mobile Robots" for and , on , Proceedings the IEEE International Conference Roboticsand Automation St. of Louis, MO , pp. 500- 05. Kim , J., and Khosla P. 1992 " Real Time Obstacle . AvoidanceUsing HarmonicPotential , " Functions IEEE Transactions Roboticsand Automation Vol. 8, No. 3, June on , , , pp. 338- 49. 's Kirchner W., and Towne W. 1994 "The SensoryBasis of the Honeybee Dance . , , " American June pp. 74- 80. , , , Language Scientific " ." , Kluge, K., and Thorpe C. 1989 Explicit Models for Robot Road Following, Proceedings on , , of the International Conference Roboticsand Automation ScottsdaleAZ , pp. 1148 54. Kohler, W. 1947 Gestalt Psychology An Introduction to New Conceptsin Modem . : , PsychologyLiveright PublishingCo., New York.

461

References Kolodner J. 1994 CaseBasedReasoningMorgan Kaufman SanMateo, CA. . , , , " PotentialField MethodsandTheir InherentLimita. KorenY., andBorenstein 1991 , , J. " tionsfor MobileRobot , of NavigationProceedings theIEEE International Conference

and . onRobotics Automation , Sacramento , pp. 13981404 , CA "Controlof . Behaviors , KoseckaJ., Bajcsy R., andMintz M. 1993 , , VisuallyGuided of Science Diversity MS - ," Technical ,U Report -CS93 101, Department Computer of Pennsylvania . , Philadelphia . AmericanVol. 268 No. 1, Kosko B., andlsaka S. 1993" FuzzyLogic" Scientific , , , , , - . , July pp. 76 81 in Substitution ." Coordination Sensory , R. Koy Oberthur 1989 Perception Sensorimotor by " in Visuomotor : Amphibians for theBlind, Coordination , Models , , Comparisons . andRobotsedsJ. P EwertandM. A. Arbib PlenumNewYork pp. 397 418 , , , , . . Avoidance ." Potential FieldApproach Obstacle to Control , Krogh B. 1984 A Generalized " SMERI Technical of MS84 , Society Manufacturing -484 , Dear , Paper Engineers . born Michigan , Path and ." , , Krogh B. andThorpe C. 1986 Integrated Planning DynamicSteering " for Vehicles Proceedings theIEEEInternational Control Autonomous , Conference of onRobotics Automation Francisco , pp. 166469. and , San , CA and . from RewardsRobotics Autonomous KroseB. 1995"Learning Delayed ," , , Systems Vol. 15 No. 4, October . 233 36. , , pp " Robotic . Kube C. R. andZhang H. 1992 "Collective , , , Intelligence FromAnimals International on 2: to Animats Proceedings theSecond of Conference Simulation of BehaviorHonoluluHI, December Press , MIT , Cambridge , pp. 460 68. , MA , , Adaptive Behaviors Collective for . Kube C. R. and Zhang H. 1994 " Stagnation , , Recovery " and on Robots Systems Robotics Proceedings theInternational , of Conference Intelligent ' ), (IROS94 pp. 188390. for ." Method RobotSpatial , , , KuipersB., andByun Y-T. 1988 A RobustQualitative " on NationalConference ArtificialIntelligence , , of Learning Proceedings theSeventh pp. 774 79. and Based ." , , Strategy Exploration Mapping KuipersB., andByun Y-T. 1991 A Robot " and of on a Semantic , Robotics Autonomous Systems Hierarchy Spatial Representations , Vol. 8, pp. 47 63. for ." , Matching Observation MultiRobot Cooperation by KuniyoshiY. 1995 Behavior " International am Research , Germany , Herrsching Ammersee , of SymposiumRobotics , pp. 343 52. ." , , K. , S. , , KuniyoshiY., NobuyukiK., Sugimoto , Nakamura , andSuehiroT. 1995 A " WideAngleLensfor ActiveVision Proceedings theIEEEInternational Foveated , of - . and on , Japan , pp. 2982 88 , May , Nagoya Conference Robotics Automation . , , , S. , , S. KuniyoshiY., Rougeaux , Ishii, M., Kit& N., Sakane , andKakikuraM. 1994 " " Task PatternsProceedings : The and , by Cooperation Observation Framework Basic on and , pp Conference Robotics Automation . 767 73. of theIEEEInternational

462

References

KutulakosK., LumelskyV., andDyer, C. 1992" Object . . , , , Exploration Purposive by " No. Science Department , Technical Dynamic , Viewpoint Adjustment Report 1124Computer of . , University Wisconsin , MadisonNovember , -Based D., Rosenblatt , andHebertM. 1994"A Behavior J. . , for , , Langer System Qff" RoadNavigation IEEE Transactions Robotics Automation . 10 No. 6, on and , , Vol , December . 776 83. , pp . . , C. (ed) 1995ArtificialLife: An Overview , Mit Press Langton , Cambridge , MA. -C. 1991Robot Latombe . MotionPlanningKluwerAcademic Publishers ,J . , , Boston Lawton D., Arkin, R. C., andCameron 1990" Qualitative J. . , , Spatial Understanding " andReactive Control Autonomous for Robots Proceedings theIEEEInternational , of ' on Robots Systems and - . Workshop Intelligent (IROS90 IbarakiJapan . 709 14 , ), , pp "The Functions Vision Modes " Lee D. 1978 . of andProcessing , , of Perceiving Information . . , eds H. PickandE. Saltzman , Wiley NewYork , Lee J. L., HuberM., DurfeeE., andKennyP 1994" UM-PRS An Implementation . : , , , , . " of theProcedural for , Reasoning System Multirobot ApplicationsProceedings the '94 of on Robotics Field FactoryandSpaceCIRFFSS ), Houston in , , Conference Intelligent ( , TX, Marchpp. 842 49. , " Lefebvre andSaridisG. 1992"A Computer . Architecture Intelligent for , D. Machines , , on and Proceedings theIEEEInternational of , Nice Conference Robotics Automation , France , pp. 274550. , May . . , A. , 2nd . , Lehninger 1975Biochemistry ed, WorthNewYork V. (ed) 1995Proceedings theInternational Lesser . . on , of Conference Multiagent Systems Francisco , June . , San , CA . for Levitt T. andLawtonD. 1990" Qualitative , , ," Navigation MobileRobots Artificial Vol. 44, No. 3, pp. 305 60. , Intelligence Lewis M., Fagg A., andBekeyG. 1994" Genetic . for in , , , Algorithms GaitSynthesis a Hexapod Robot in Recent Trends MobileRobotsed Y. ZhengWorldScientific in ," , . , , - . , pp Singapore . 317 31 ." for MobileRobots Proceedings Lim W. 1994 An AgentBased , ," Approach Programming on and of theIEEEInternational , San , Conference Robotics Automation Diego CA, pp. 358489. Lin, L. 1992 " SelfImproving . Reactive Based Reinforcement on , Agents Learning " and - . , , Planning TeachingMachine LearningVol. 8, pp. 293 321 -Handling anObjectSorting Lin, F. andHsu J. 1995"Cooperation Deadlock . and for , " Taskin a MultiAgent Robotic Conference , System Proceedings theIEEEInternational of onRobotics Automation . 258085. and , pp " ." SimulatorsScience . 273 pp. 107378. , , , Vol , Quantum Lloyd S. 1996 Universal -VerlagNewYork LorenzK. 1981TheFoundations EthologySpringer . . , , , of LorenzK., andLey hausen . 1973Motivation Human AnimalBehaviorAn . and : , ,P of View Reinhold , NewYork Co. . Ethological , VanNostrand " Lumia R. 1994" UsingNASREM RealTIlDe . for Interactive Robot Control , , Sensory - 35 Robotica . 12 pp. 127 . , Vol ,

463

References

" . Lund, J., Wu, Q., andLevitt, J. 1995 " VisualCortexCell TypesandConnection in The , Handbookof Brain TheoryandNeuralNetworks ed. M . Arbib , MIT PressCambridge , , , MA , pp. 344 48. " ." , , , , Lyons D. 1992 Planning Reactive Encyclopediaof Artificial Intelligence ed. S. , , Shapiro 2nd ed., JohnWiley andSons New York, pp. 1171 82. ." , Lyons D., and Arbib, M. 1989 A Formal Model of Computationfor SensoryBased " IEEE Transactions Roboticsand Automation Vol. 6 No. 3 June . on Robotics , , , , pp , 280- 93. " ." , , , Lyons D., and Hendriks A. 1992 Planningfor ReactiveRobot Behavior Proceedings on , , of the IEEE International Conference Roboticsand Automation Nice, France pp. 2675 80. " ." , , , Lyons D., and Hendriks A. 1993 SafelyAdapting a HierarchicalReactiveSystem and Proceedings SPIEIntelligent Robotsand ComputerVisionXII : Active VISion 3D of Methods Vol. 2056 pp. 450- 59. , , " : ;" , , , Lyons D., and Hendriks A. 1994 Planningby Adaptation ExperimentalResults on , Proceedings the IEEE International Conference Roboticsand Automation San of --60 Diego, CA , pp. 855 . ." , , Lyons D., and Hendriks A. 1995 Planningas IncrementalAdaptationof a Reactive " RoboticsandAutonomous , , SystemsVol. 14, No. 4, pp. 255 88. System of . MacKenzie D. 1996 "A DesignMethodologyfor the Specification BehaviorBased , " Robotic Systems PhiD. Dissertation College of Computing Georgia Institute of , , , , TechnologyAtlanta. : . MacKenzie D., and Balch, T. 1993 " Making a Clean Sweep Behavior BasedVacuuming , " : , , , in working notes AAAI Fall Symposium InstantiatingReal world Agents , , Raleigh NC, October pp. 93- 98. and . MacKenzie D., CameronJ., andArkin , R. 1995 " Specification Executionof Multiagent , , on Missions" Proceedings the International Conference Intelligent Robotics , of and SystemsIROS ' 95), Pittsburgh PA, pp. 51- 58. , ( . MacLennan B. 1991 " SyntheticEthology: An Approachto the Study of Communication , " of ! , , in Artificial life II , SF Studiesin the Sciences Complexity Vol. XI , ed. Farmeret aI., Addison Wesley Reading MA . , , " Maes P. 1989 "The Dynamicsof Action Selection Proceedings the EleventhInternational . , of on Joint Conference Artificial Intelligence(IJCAI-89), Detroit, MI , pp. 99197. . Maes P. 1990 " SituatedAgentsCan HaveGoals" RoboticsandAutonomous , , Systems , Vol. 6, pp. 49- 70. " Behaviors Proceedings the . Maes P., andBrooks R. 1990 " Learningto Coordinate , , of , '90 Boston MA on , August , , EighthNational Conference Artificial Intelligence(AAAI ), pp. 796- 802. . Mahadevan S., and Connell J. 1991 "Automatic Programmingof BehaviorBased , , ' RobotsUsing Reinforcement : of Learning Proceedings theNinth National Conference on Artificial Intelligence(AAAI ' 91), Anahelm CA , July, pp. 768- 73. ,

464

References Malcolm, C., and Smithers T. 1990 " Symbol Grounding via a Hybrid Architecture . , " in an AutonomousAssemblySystem in DesigningAutonomous , , , Agents ed. P. Maes MIT Press Cambridge MA , pp. 123 44. , , Malkin, P. and Addanki, S. 1990 " LOGnets A Hybrid GraphSpatialRepresentation . : for Robot Navigation" .Proceedings the Eighth National Conference Artificial on , of , Intelligence(AAAI-90), Boston MA , pp. 1045 50. ." , , MarapaneS., Holder M., andTrivedi, M . 1994 Coordinating Motion of Cooperative " Mobile Robots through Visual Observation Proceedings the IEEE International , of on , Man and CyberneticsSanAntonio, TX , pp. 2260 65. , Conference Systems Marr, D. 1982 Vision A Computational . : Investigationinto the HumanRepresentation and Processing VisualInformalion, W.H. FreemanSanFrancisco CA. , of , Mataric, M. 1990 " Navigatingwith a Rat Brain: A Neurobiologicallyinspired Model . " for Robot SpatialRepresentation FromAnimals to Animats Proceedings the First : , of International Conference Simulationof AdaptiveBehavior, MIT Press Cambridge on , , MA , pp. 169 75. Mataric, M . 1992a " BehaviorBasedConttol: Main Properties Implications" Proceedings . and , on Intelligent Control Systems International Conference on , of Workshop Roboticsand Automation Nice, France May. , , Mataric, M. 1992b " Integrationof Representation Goal-Driven BehaviorBased . into Robots" IEEE Transactions RoboticsandAutomation Vol. 8, No. 3, June pp. 304on , , , 12. Mataric, M . 1992c " Minimizing Complexity in Conttolling a Mobile Robot Population . " on , IEEE International Conference Roboticsand Automation Nice, France pp. , , 830- 35. " Mataric, M. 1993a " Synthesizing . on , Group Behaviors Proceedings the Workshop of on , Dynamically Interacting Robots International Joint Conference Artificial Intelligence (IJCAI-93), ChamberyFrance August pp. 1- 10. , , , Mataric, M. 1993b " Kin Recognition Similarity, andGroupBehavior" Proceedings . , , of - . the FifteenthAnnual CognitiveScienceConferenceBoulder CO, June pp. 705 10 , , , Mataric, M . 1994a " Interactionand Intelligent Behavior" PhiD. Dissertation Department . , , of ElectricalEngineering ComputerScienceMassachusetts and Instituteof Technology , , Cambridge May. , Mataric, M. 1994b " Learning to BehaveSocially" FromAnimals to Animats 3: Proceedings . , on of the Third International Conference Simulationof Adaptive Behavior, , Bright on UK , pp. 453- 62. Matthies L., andElfes, A . 1988 " Integrationof SonarandStereoRangeDataUsing a . , " Grid-BasedRepresentation Proceedings the International Conference Robotics on , of . and Automation Philadelphia PA, April , pp. 727 33. , , Mazokhin PorshnyakovG. 1969 Insect Vision PlenumPress New York. . , , , " " . , , McCarthy J. 1995 Making Robots Consciousof their. Mental States Computer Science , , Report StanfordUniversity CA , July 24. . , , , McCarthy J., Minsky, M., RochesterN., and Shannon C. 1955 A Proposalfor the DartmouthSummer Research , Project on Artificial Intelligence August 31.

465

References

. McCulloch, W., and Pitts, W. 1943 "A Logical Calculus of the Ideas Immanentin NervousActivity : ' Bulletin of MathematicalBiophysics Vol. 5, pp. 115 33. , McFarland D. 1981 The Oxford Companionto Animal Behavior, Oxford University . , Press . . McFarland D., and Bosset U. 1993 Intelligent Behaviorin Animalsand Robots MIT , , , Press Cambridge MA . , , . Locomotion" Simulation pp. McGhee R. 1967 " Finite StateControl of Quadruped , , , . 135 40, September . McKerrow P. 1991 Introductionto Robotics Addison Wesley Reading MA . , , , , " . , Meystel A. 1986 Planning in a Hierarchical Nested Controller for Autonomous -fifth Conference Decisionand Control, Athens Robots" Proceedings the 1Wenty on , , of Greece pp. 1237 49. , Milian , J. 1994 " Learning Efficient ReactiveBehavioralSequences . from Basic Re-DirectedAutonomousRobot" (FromAnimalsto Animats3), Proceedings flexesin a Goal , on , of the Third Conference Simulationof AdaptiveBehavior, MIT Press Cambridge , MA , pp. 266- 74. " Miller , D. 1995 " Experiences . , , Looking into Niches WorkingNotes 1995AAAI Spring : LessonsLearnedfrom Implemented SoftwareArchitectures Physical for Symposium , Agents PaloAlto ~CA, March, pp. 141 45. . Miller , E., andDesimoneR. 1994 ScienceVol. 263, Janury pp. 520- 22. , , . Miller , G., Galanter E., and Pribram K. 1960 Plans and the Structureof Behavior, , , Holt, Rinehart andWinston New York. , , . , Minsky, M . 1986 TheSocietyof theMind , Simonand SchusterNew York. " Will RobotsInherit the Earth? " Scientific American Vol. 271, No.4, . , Minsky, M . 1994 October pp. 109 13, , . : , , Minsky, M ., and Papert S. 1969 PerceptronsAn Essayin ComputationalGeometry MIT Press Cambridge MA . , , . Mishkin, M ., UngergleiderL., andMacko, K. 1983 " ObjectVision andSpatialVision: , " ' l \voCortical Pathways Trendsin Neurosciences , , Vol. 6, pp. 414- 17. " Mitchell, T. 1990 " BecomingIncreasinglyReactive Proceedings the Eighth National . , of on , Conference Artificial Intelligence(AAAI-90), Boston MA , pp. 1051 58. . T., Araj , F., Tadashi H., and Idogaki, T. 1996 " SelfOrganizing Mitsumoto, N., Fukuda , , " Robotic System Proceedingsof the IEEE International Conference , Multiple on RoboticsandAutomation Minneapolis MN , April , pp. 1614 19. , , H. 1983 " Introduction into Cyberneticsof Orientation Behavior" Biophysics . , Mittelstaedt , . , , eds W. Hoppeet al., SpringerVerlag Berlin, pp. 794- 801. " of . Mittelstaedt H. 1985 "Analytical Cybernetics SpiderNavigation Neurobiologyof , , Arachnids ed. F. Barth, SpringerVerlag Berlin, pp. 298- 316. , , " ." , , , Mizessyn F., and Ishida Y. 1993 Immune Networks for CementPlants Proceedings on Decentralized , , SystemsKawasaki of theInternationalSymposium Autonomous , Japan pp. 282 88.

466

References

. MochidaT., Ishiguro A., Aoki, T., andUchikawaY. 1995 " Behavior , , Arbitration , for Autonomous Mobile Robots :' UsingEmotionMechanismsProceedings the of ' IEER S International E J on Robots Systems and Conference Intelligent (IROS 95 ), , PA Pittsburgh , pp. 516 21. . and Control Mondada , andFloreano 1995"Evolution Neural , F. St11lctures : Some , D. " on and , Experiments MobileRobots Robotics Autonomous , Vol , Systems . 16 No. 2-4, -96. pp. 183 -Based ." , J. , , Montgomery , FaggA., andBekeyG. 1995 The USCAFV l : A Behavior " Intemationa ] AerialRobotics Entryin the 1994 , , , CompetitionIEEEExpert Vol. 10 No. 2, April, pp. 16 -22. -Based MooreA., AtkesonC., andSchaal , 1995" Memory . for , , , S. ," Learning Control -RI-TR-95 18 Carnegie - , -MellonUniversity CMU Robotics Institute Technical CMU Report , Pittsburgh , April. , PA Morasso ., andSanguineti 1994"SelfOrganising -Schema MotorPlanning . for ,P , V. Body " , Journal MotorBehaviorVol. 26, pp. 131 48. , of " MoravecH. 1977"Towards . Automatic VisualObstacle AvoidanceProceedings , , of the Fifth International Joint Conference Artificial Intelligence on , Cambridge , , MA . , Augustp. 584 " Moravec 1983"The Stanford andtheCMURover Proceedings theIEEE . Cart , H. , , of Vol. 71 No. 7, pp. 872 84. , " MoravecH. 1985" Robots Rove in Autonomous . that MobileRobots Annual , , , Report -MellonUniversity Technical No. CMU RI-TR-86 , Robotics -4 InstituteCarnegie , Report . , Pittsburgh , February , PA MoravecH. 1988 Mind Children TheFutureof Robotand HumanIntelligence . : , , Harvard Press , Cambridge , MA. University MoravecH. Forthcoming . MindAge Transcendence : RobotsOxfordUniversity , , through Press . MoravecH., andElfes A. 1985"High Resolution . , , ," Mapsfrom WideAngleSonar theASME International in Proceedings of , Boston , Computers Engineering Conference
." , , , , and Moynihan M. 1970 Control, SuppressionDecay Disappearance Replacement of Displays" Journal of TheoreticalBiology, Vol. 29, pp. 85- 112 . , ." Murphy, R. R. 1991 An Application of DempsterShaferTheory to a Novel Control " Schemefor SensorFusion Proceedings SPIE Stochastic Methodsin Signal Processing , of , ImageProcessingand ComputerVision SanDiego, CA , July, pp. 55- 68. , , " ." , Murphy, R. R. 1992 An Architecturefor Intelligent Robotic SensorFusion PhiD. -ICS-92 , College of Computing Georgia Dissertation TechnicalReport No. GIT /42 , , Instituteof TechnologyAtlanta. , ." : for Murphy, R. R., and Arkin , R. C. 1992 SFX An Architecture ActionOriented MA , pp. 375- 80.

" Sensor Fusion Proceedings theInternational on Robotics , of Conference Intelligent ' andSystems , (IROS92 RaleighNC, July pp. 107986. ), , ." Guided Stable with , , , A. MurphyT., LyonsD., andHendriks 1993 Visually Grasping " -Based a Multi-Fingered Robot HandA Behavior : , Approach Proceedings theSPIE of

467

References

and , , Intelligent Robotsand ComputerVisionXII : Active VISion 3D Methods Vol. 2056 . 252 63. pp Nauta W., and Feirtag M. 1979 "The Organization the Brain," in TheBrain, W.H. . of , , FreemanSanFranciscopp. 40- 55. , , Nehmzow U., and McGonigle B. 1994 "Achieving Rapid Adaptationsin Robotsby . , , Means of External Tuition," From Animals to Animats 3: Proceedings the Third of on , , Conference Simulationof AdaptiveBehavior, Mit PressCambridge MA , pp. 301 08. NehmzowU., Smithers T. andMcGonigle B. 1993 " Increasing . BehavioralRepertoire , , , in a Mobile Robot" From Animals to Animats 2: Proceedings the SecondInternational , of on , Conference Simulationof Adaptive Behavior, Honolulu, ffi , Mit Press , Cambridge MA , pp. 291 97. Neisser U. 1976 Cognition and Reality Principles and Implications of Cognitive . : , . , , PsychologyW.H. FreemanSanFrancisco " Neisser U. 1989 " Direct Perception Recognitionas Distinct Perceptual . and , , Systems text of address to , presented the CognitiveScience Society August. " Without Neisser U. 1993 . : , , PerceptionThereIs No Knowledge Implicationsfor Artificial " , Intelligence in Natural and Artificial Minds, ed. R. G. Burton, StateUniversity of New York Press Albany, pp. 147 64. , " Visual Scene " Nelson J. I. 1995 . : , , Perception Neurophysiology in The Handbookof Brain Theoryand Neural Networks ed. M. Arbib , Mit Press Cambridge MA , pp. , , , 1024 28. Neumann 0 ., and Prinz, W. 1990 " Prologue Historical Approach to Perception . : es , and Action," in Relationships betweenPerceptionand Action, eds O. Neumannand . W. Prinz, SpringerVerlag Berlin, pp. 5- 19. , Nilsson, N. 1965 Learning Machines Foundations TrainablePattern Classifying . : of SystemsMcGraw-Hill , New York. , Nilsson N. 1969 "A Mobile Automaton An Application of Artificial Intelligence . : , " on , of Techniques Proceedings the International Joint Conference Artificial Intelligence JJCAI 69) . WashingtonD.C., May. Reprintedin Autonomous Mobile Robots , ( . Vol. 2, eds S. IyengarandA . Elfes, IEEE ComputerSocietyPress Los Alamitos, 1991 , , pp. 233- 44. . Nilsson N. 1980 Principles of Artificial Intelligence Tioga PaloAlto , CA. , , , " TechnicalNote No. 323 Artificial " . Nilsson N. 1984 Shakeythe Robot , , , Intelligence Center SRI International Menlo Park CA. , , , . for Nilsson N. 1994 "Teleo-ReactivePrograms Agent Control," Journal of Artificial , , IntelligenceResearchVol. 1, pp. 139 58. Nilsson N. 1995 " Eye on the Prize," AI Magazine Vol. 16, No. 2, Summer pp. 9- 17. . , , , F., and Chatila R. 1989 " Control of Mobile Robot Actions," Proceedings . Noreils, , of on the IEEE International Conference Roboticsand Automation pp. 701 07. ,

468

References

Noreils, F., andChatila R. 1995 " PlanExecutionMonitoring andControl Architecture . , for Mobile Robots" IEEE Transactions Roboticsand Automation Vol. 11, No. 2, on , , , pp. 255 66. April NormanD ., andShallice T. 1986 "Attention to Action: Willed andAutomaticControl . , of Behavior" in Consciousness Self-Regulation Advances Research Theory and : in and , , Vol. 4, eds R. Davidson G. Schwartz and D. Shapiro PlenumPress New York pp. . , , , , , 1- 17. Noton D., and Stark L . 1971 " Scanpaths Saccadic . in While Viewing , Eye Movements " andRecognizing Patterns WsionResearchVol. 11, pp. 929- 42. , , OlshausenB., and Koch, C. 1995 " SelectiveVisual Attention," in The Handbookof . , Brain Theoryand Neural Networks ed. M . Arbib, Mit Press Cambridge MA , pp. , , , 837 40. Ooka A., Ogi, K., Wada Y., Kida, Y., Takemoto A., Okamoto K., and Yoshida K. , , , , , . 1985 " Intelligent Robot Systemn ," RoboticsResearchSecond International Symposium , . and , eds H. Hanafusa H. Inoue Mit Press Cambridge MA , pp. 341 47. , , , " K., and Eklundh J. 1993 " Heads Eyes and Head Eye Systems in Active Pahlavan . , , , , Robot Vision CameraHeads Model-BasedNavigation and Reactive Control, eds : . , H. ChristensenK. Bowyer andH. Bunke, World Scientific Singaporepp. 33- 49. , , , , " Pahlavan K., Uhlin , T., and Eklundh, J. 1993 "Active Vision as a Methodology in . , , Active Perceptioned. Y. Aloimonos LawrenceErlbaumAssociatesHillsdale, NJ, pp. , , , 19 46. -

. PaniJ. 1996Personal communication . , , May "Localversus " Parker 1992 . Global Control Lawsfor Cooperative TeamsAI , L. , Agent Me morandum 1357 MIT AI Lab Cambridge , March No. . , , , MA " ParkerL. 1994 "Heterogeneous -RobotCooperationPhiD Dissertation . Multi . , , , De of and Science Institute of , Massachusetts parbnent Electrical Engineering Computer . , Cambridge , February , MA Technology ParkerL. 1995"Alliance An Architecture FaultTolerant . : for MultiRobotCooperation , National Technical morandum 12920February Me No. . ," OakRidge , Laboratory PatelM., Colombetti , andDorigoM. 1995"Evolutionary M. . for Intelligent , , , Learning " :A Automation Case Automation SoftComputing . I , No. I , and , , Vol Study Intelligent pp. 29 42. " . in PauL. 1991" Behavioral / Data , Knowledge Sensor Fusion Systemsin ActivePerception -VerlagBerlin pp. and andRobotVisioneds A. Sood H. Wechsler , . , Springer , , 357 72. . Press . Pavlov1 1927Conditioned , . , OxfordUniversity , London Reftexes "Internalized : A " . Plans Representation ActionResources Designing for , , in PaytonD. 1991 ed P MaesMIT Press . . MA, pp. 89 103 Autonomous - . , , , Cambridge , Agents ." , , D. , , , J. PaytonD., Keirsey , Kimble D., Krozel J., andRosenblatt 1992 Do Whatever " -tolerant : to Autonomous Works A Robust Control AppliedIntelligence , Approach Fault , Vol. 2, No. 3, September . 225 50. , pp American . 235 pp. 72 86. Pearson 1976"The Control Walking Scientific . of ," , Vol , , K.

469

References Pellionisz A., and ilinas , R. 1980 "Tensorial Approach to the Geometryof Brain . , " Function CerebellarCoordination via a Metric Tensor Neuroscience Vol. 5, pp. : , , 1125 36. PenroseR. 1989 TheEmperor's NewMind , Oxford University Press New York. . , , PenroseR. 1994 Shadows / the Mind , Oxford University Press New York. . a , , . : , Piaget J. 1971 Biology and Knowledge An Essayon the RelationsbetweenOrganic and es . Regulations CognitiveProcess , University of ChicagoPress Pin, F., and Watanabe Y. 1995 "Automatic Generationof Fuzzy Rules Using the . , " : , FuzzyBehavioristApproach The Caseof SensorBasedRobotNavigation Intelligent Automationand Soft Computing Vol. 1, No. 2, pp. 161 78. , PomerleauD. 1993 Neural NetworkPerception or Mobile Robot Guidance Kluwer . , , / AcademicPublishersBoston . , PomerleauD. 1995 " Ralph: Rapidly AdaptingLateralPositionHandler" Proceedings . , , 0 the IEEE Symposium Intelligent VehiclesDetroit, MI , Septemberpp. 506- 11. on , , / Premvuti S., andYuta S. 1990 " Consideration theCooperation Multiple Autonomous . on of , , Mobile Robots" IEEE International Workshop IntelligentRobotsand Systems on , ' , (IROS 90), Tsuchiura Japanpp. 59- 63. P., Swain M., and Kahn, R. 1994 "Task and EnvironmentSensitive . , , Prokopowicz " on , Proceedings the IEEE Workshop VisualBehaviors Seattle WA, June , , , Tracking of pp. 73- 78. ." , Quinn, R., and EspenschiedK. 1993 Control of a HexapodRobot using a Biologically " in , InspiredNeural Network Biological Neural Networksin InvertebrateNeuand . , , , , , roethology Robotics eds R. Beer R. Ritzmann andT. McKenna AcademicPress SanDiego, CA, pp. 365- 81. ." Ram A., Arkin , R., Boone G., and Pearce M. 1994 Using GeneticAlgorithms to , , , Learn ReactiveControl Parameters AutonomousRobotic Navigation" Journal of for , AdaptiveBehavior, Vol. 2, No. 3, pp. 277 305. ." Ram A ., Arkin , R. C., Moonnan K., and Clark R. 1997 CaseBased Reactive , , , -BasedMethod for On-Line Selectionand Adaptationof Reactive : Navigation A Case " in Control Parameters Autonomous Robotic Systems IEEE Transactions Systems on , , . Man and CyberneticsPartB , Vol. 27. No. 3, pp. 376- 94. , . Reece D., andShafer S. 1991 "Active Vision at the SystemLevel for RobotDriving," , , on , , Working Notes AAAI Fall Symposium SensoryAspectsof Robotic Intelligence Asilomar, CA , Novemberpp. 70- 77. , ." IncrementalLearning of Fuzzy Rules" Roboticsand , , Reignier P. 1995 Supervised Autonomous Vol. 16, pp. 57- 71. , Systems " : ." , , , Reynolds C. 1987 Flocks Herds and Schools A Distributed BehavioralModel, --34. , ComputerGraphics Vol. 21, No. 4, pp. 25 " ." , , , Rimey R. 1992 Task-OrientedVision with Multiple BayesNets in Active Vision eds A. Blake, andA. Yuille, MIT PressCambridge MA . . , ,

470

References " Riseman E., and Hanson A . 1987 " GeneralKnowledge BasedVision Systems in . , , , . Vision Brain and Cooperative , , , , Computationeds M. Arbib andA. Hanson MIT Press , Cambridge MA , p. 287. ." ron: Rosenblatt F. 1958 The Percept A ProbabilisticModel for Information Storage , andOrganization the Brain," Psychological in Review Vol. 65, pp. 386- 408. , ." Rosenblatt J. 1995 DAMN : A Distributed Architecture for Mobile Navigation" , , on Learnedfor Implemented , Working Notes AAAI 1995 Spring Symposium Lessons Architectures PhysicalAgents PaloAlto , CA , March, pp. 167 78. , Software for "A Fine-GrainedAlternativeto the . Rosenblatt J., and Payton D. 1989 , , Subsumption Architecturefor Mobile RobotControl," Proceedings the InternationalJoint Conference of on Neural Networks June pp. 317 23. , , RosenscheinS., and Kaelbling, L . 1987 "The Synthesisof Digital Machineswith . , " Provable NoteNo. 412, Menlo Park , , EpistemicProperties SRIInternational Technical CA, April . ." Rumelhart D., Hinton, G., and Williams, R. 1986 LearningInternal Representations , " in Parallel Distributed . and , , by Error Propagation ProcessingVol. I , eds D. Rumelhart J. McClelland MIT PressCambridge MA , pp. 318- 62. , , , Russell A., Thiel, D., and Mackay Sim, A. 1994 " SensingOdour Trails for Mobile . , RobotNavigation" Proceedings theIEEE International Conference Roboticsand on , of Automation SanDiego, CA, May, pp. 2672 77. , " Sacerdoti E. 1974 " Planningin a Hierarchyof AbstractionSpaces Artificial Intelligence . , , , Vol. 5, No. 2, pp. 115 35. SacerdotiE. 1975 "A Structurefor PlansandBehavior" PhiD. DissertationTechnical . , , , Note No. 109 AI Center SRI International Menlo Park CA. , , , , Saffiotti, A., Ruspini E., and Konolige, K. 1993a " Blending Reactivity and Ooal. , Directedness a Fuzzy Controller" Second in IEEE International Conference Fuzzy on , , , SystemsSanFrancisco CA, March, pp. 134 39. Saffiotti, A., Ruspini E., and Konolige, K. 1993b " RobustExecutionof Robot Plans . , " on , Proceedings the Workshop FuzzyLogic in Artificial Intelligence Using Fuzzy Logic of on / ' , InternationalJoint Conference Artificial Intelligence(IJCA 93), Chambery , France pp. 24- 37. , Saffiotti, A , Konolige K., and Ruspini E. 1995 "A Multi -ValuedLogic Approachto . , , " , IntegratingPlanningand Control, Artificial Intelligence Vol. 76, No. 1-2, July, pp. 481- 526. Sahota M . 1993 " Real Time Intelligent Behaviorin DynamicEnvironments Soccer . : , " of , , , Playing Robots MiS. Thesis Department ComputerScience University of British . Columbia Vancouver , Saito F., Fukuda T., and Araj, F. 1994 " Swing and Locomotion Control for a lWo. , , Link BrachiationRobot" IEEE Control Systems , , , Magazine Vol. 14, No. 1, February pp. 5- 11. " Sanborn J. C. 1988 "A Model of Reactionfor Planningin Dynamic Environments . , , MiS. Thesis ComputerScienceDepartment University of Maryland College Park , , , , May.

471

References

." an with Sarachik 1989 Characterizing IndoorEnvironment a MobileRobotand , K. " on Uncalibrated Stereo Proceedings theIEEEInternational , of Conference Robotics -89. andAutomation , Scottsdale , pp. 984 , AZ " . Robotic Control IEEETransactions Automatic on Control SaridisG. 1983" Intelligent , , - 56. , , Vol. 28, No. 5, May pp~547 Valvanis 1987" OntheTheory Intelligent . of Controls Proceedings SaridisG., and ," , , K. Robotic on in , Cambridge , , MA Systems of theSPIEConference Advances Intelligent October . 488 , pp --95. ' . ?" , Vol , Schank 1987"What sAI, Anyway AI Magazine . 8, No.4, Wmterpp. 59 65. , R. " " in TheHandbook Brain . and , N. , Theory Neural of Schmajuk 1995 Cognitive Maps . Networks . M. Arbib MIT Press , , Cambridge , pp. 197 200 , MA , ed MotorSkill Learning Psychological . of ," Schmidt 1975"A Schema , R. Theory Discrete Review . 82 pp. 225 60. , Vol , -SchonerG., andDose M. 1992 "A Dynamical to . , , Systems Approach TaskLevel " Autonomous Vehicle Motion Robotics Used PlanandControl to , System Integration - . andAutonomous , Vol , Systems . 10 pp. 253 67 " Universal Robots Unpredictable in Environments . Plans Reactive for , M. Schoppers 1987 JointConference ArtificialIntelligence on International ," Proceedings theTenth of - ), (IJCAI87 pp. 852 59. " " In Defense Reaction Plans CachesAI Magazine . 10 as . of , Vol , , , M. Schoppers 1989 No.4, Winterpp. 51 60. , to Behaviors . SchultzA., andGrefenstette 1992"Usinga Genetic , J. , Algorithm Learn " NCARAITR #AIC-92-009 Vehicles NavalResearch for Autonomous , , Laboratory , DC. Washington Howto to . Trick: UsingClassical ScuttT. 1994"The FiveNeuron , Conditioning Learn " 3: Animals Animats ProceedingstheThirdConfonSimulation to Seek , of . Light From -70. BehaviorMIT Press , MA , , Cambridge , pp. 364 ofAdaptive " and . Searle 1980"Minds BrainsandProgramsBehavioral BrainSciences . 3, , Vol , , , J. , pp. 417 57. -Based : ." , , G. LearningAcquisition Manipulator SegreA., andDeJong 1985 Explanation " 62 Observation of Planning , , Working Group Paper , AI Research Ability through -Champaign h. . M~ at Lab of Science , Univerity Dlinois Urbana Coordinated ," American . Neisser 1960"Pattern , ,0 , U. Recognition Machine Scientific by Selfridge ., and , Vol. 203 pp. 60 68. , -Ivaldi F. 1994" Geometric Controller Structure theAdaptive of . Shadmehr , andMussa , , R. . No. of theHuman , " MIT AI Memo 1437Cambridge , March , MA , Arm of . Behavior Robots ShibataT., Ohkawa K., and Tanie K. 1996 " Spontaneous , , , " RobotSystem Proceedings the IEEE for CooperationEmotionally , of Intelligent on and International , MN , Minneapolis , April, pp. Conference Robotics Automation 242631. Information led Human . Shiffrin R., andSchneider 1977"Control andAutomatic , W. , Review . 84 pp. 127 90. : II ," Psychological , Vol , Processing

472

References

." Microrover" Autonomous Robots ShirleyD ., andMatijevic, J. 1995 Mars Pathfinder , , Vol. 2, pp. 283- 89. Shor P. 1994 "Algorithms for Quantum . : , Computation DiscreteLogarithmsandFactoring " on , Proceedings the Thirty-fifth AnnualSymposium theFoundations Computer of of ScienceSantaFe, NM , Novemberpp. 124 34. , , Simmons R. (ed.) 1992 Working notes AAAJ 1992 Spring Symposium Control of . on , , Selective , , PerceptionStanfordUniversity March, pp. 138 41. Simmons R., and Koenig S. 1995 " ProbabilisticRobot Navigationin Partially Observable . , , " Environments Proceedings theInternationalJoint Conference Artificial on , of , , Intelligence(IJCAI-95), Montreal CA , August pp. 1080 87. Simon H. 1983 " Why ShouldMachinesLearn?" in MachineLearning: An Artificial . , , IntelligenceApproach Vol. 1, cds. R. Michalski, J. Carbonell and T. Mitchell, Tioga , , Pubishing PaloAlto , CA , pp. 25- 39. " Sims K. 1994 " Evolving Virtual Creatures Proceedings theACM Siggraph94, pp. . , , of 15- 22. Skinner B.F. 1974 About Behaviorism Alfred Knopf, New York. . , , ." Slack M . 1990 SituationallyDriven Local Navigationfor Mobile Robots" JPL Publication , , No. 90 17, NASA Jet PropulsionLaboratory Pasadena , April . , , CA Smith, W. J. 1977 TheBehaviorof CommunicatingAn EthologicalApproach Harvard . : , Press Cambridge MA . , , University Soldo M . 1990 " Reactive Preplanned . and Control in a Mobile Robot" Proceedings , , of the IEEE International Conference Roboticsand Automation Cincinnati, OH, pp. on , 1128 32. " ." in , , Spector L. 1992 Supervenience Dynamic World Planning PhiD. Dissertation , UMIACS-TR-92-55, University of Maryland CollegePark . , ." on Spinelli, D. N. 1987 A Traceof Memory: An EvolutionaryPerspective the Visual " in , Vision Brain and Cooperative , , System , Computationcds. M . Arbib andA . Hanson MIT Press Cambridge MA , pp. 165 --82. , , ." Stark L ., and Bowyer K. 1991 Achieving Generalized , , Object Recognitionthrough " aboutAssociationof Functionto Structure IEEE Transactions Pattern on , Reasoning . , , Analysisand MachineIntelligence Vol. 13 No. 10, October pp. 1097 1103 , " Function BasedGeneric . Stark L., and Bowyer K. 1994 , , Recognitionfor Multiple " : , , , ObjectCategories CVGIP Image UnderstandingVol. 59, No. 1, Januarypp. 1- 21. ." Revisited Cognitive Models Direct Active : Stark L., and Ellis, S. 1981 Scanpaths , " : , , Looking, in Eye Movements Cognition and Visual Perception cds. C. Fisher R. Monty andJ. SendersLawrenceErlbaumAssociatesMahwah NJ, pp. 193 226. , , , " Steels L . 1990 " Exploiting Analogical Representations in DesigningAutonomous . , , , , , , Agents ed. P. Maes MIT Press Cambridge MA , pp. 71- 88. Steels L. 1994 " EmergentFunctionality in Robotic Agents through On-Line Evolution . , " on , Proceedings Artificial life IV , MIT Press Cambridge MA , pp. 8- 14. , , Steels L. 1995 " When Are RobotsIntelligent AutonomousAgents " Roboticsand . ? , Autonomous , SystemsVol. 15, pp. 3- 9.

473

References . Stein L. 1994 " Imagination and SituatedCognition" Journal of Experimentaland , , AI Theoretical , Vol. 6, No. 4, October Decemberpp. 393 407. , Stentz A. 1994 " Optimal and Efficient Path Planningfor Partially-Known Environments . , " on , Proceedings the IEEE International Conference Roboticsand Automation of , May, SanDiego, CA, pp. 3310 17. Stentz A ., andHebert M. 1995 "A CompleteNavigationSystemfor Goal Acquisition . , , " in UnknownEnvironments Proceedings theI EEFl R J International Conference S on , of ' , ( Intelligent Robotsand SystemsIROS 95), Pittsburgh PA, pp. 425- 32. Stone H. (ed.), 1980 Introductionto Computer . Architecture 2nd ed., SRA, Chicago . , , " . Stroulia E. 1991 " ReflectiveSelf-Adaptive Systems PhiD. Dissertation Collegeof , , , , , , Computing GeorgiaInstituteof TechnologyAtlanta GA. " ." : , , Suga N., and Kanwal, J. 1995 Echolocation CreatingComputational Maps in The Handbookof Brain TheoryandNeuralNetworks ed. M. Arbib, MIT PressCambridge , , , .MA , pp. 344 48. ." , , Sugihara K., and Suzuki I. 1990 DistributedMotion Coordinationof Multiple Mobile on Robots" Proceedings the Fifth InternationalSymposium Intelligent Control, , of , PA, pp. 138 43. Philadelphia SussmanG. 1975 A ComputerModel of Skill Acquisition, American Elsevier New . , , York. " . Sutton R. 1988 " Learning to Predictby the Methodsof TemporalDifferences Machine , , Learning, Vol. 3, pp. 9- 44. . Tachi S., and Komoriya K. 1985 " Guide Dog Robot" RoboticsResearch Second , , , , InternationalSymposiumeds H. Hanafusa H. Inoue MIT PressCambridge MA , . and , , , , pp. 333 40. . Takeuchi S. 1996 " Hybrid InsectRobot" World Wide WebURL , , .ac. .html, University of Tokyo. . .t.u /abstract http:// scorpioleopard -tOkyo jp /takeuchi . Tan M. 1991 " Cost SensitiveRobot Learning" PhiD. Dissertation TechnicalReport , , , CMU-CS-91- 134 School of Computer Science CarnegieMellon University Pittsburgh , , , , PA, May. Tanimoto S. 1990 The Elementsof Artificial Intelligence ComputerSciencePress . , , , New York. . Thorndike E. 1911 Animal Intelligence Hafner Darien CT. , , , , " "An . Thrun, S. 1995 , Approachto Learning Mobile Robot Navigation Roboticsand 4, pp. 301 20. Autonomous , SystemsVol. 15, No. . Tianmiao W., and Bo, Z. 1992 "Time -Varying PotentialField Basedon Perception , Action Behaviorsof Mobile Robot" Proceedings the 1992IEEE International Conference , of on Roboticsand Automation Nice, France pp. 2549 54. , , . , , Tinbergen N. 1953 SocialBehaviorin Animals, Methuen London. . Tsai, W., and Chen Y. 1986 "Adaptive Navigationof AutomatedVehiclesby Image , " on , , , , Analysis Techniques IEEE Transactions SystemsMan and CyberneticsVol. 16 No. 5, Septemberpp. 730- 40. ,

474

References " Tsotsos J. 1989 "The Complexity of Perceptual . SearchTasks Proceedings the , , of EleventhInternationalJoint Conference Artificial Intelligence(IJCAI '89), Detroit, on MI , pp. 1571 77. Tsotsos J. 1990 "Analyzing Vision at the Complexity Level," Behavioral and Brain . , SciencesVol. 13, pp. 423- 69. , " Tsotsos J. 1992 " On the RelativeComplexityof Active versusPassive . Visual Search , , InternationalJournal of ComputerVision Vol. 7, No. 2, pp. 127 41. , Tsotsos J. 1995 " BehavioristIntelligenceand the ScalingProblem" Artificial Intelligence . , , , Vol. 75, No. 2, June pp. 135 60. , Thrban E. 1992 Expert Systems Applied Artificial Intelligence MacMillan, New . and , , York. " ." , Thring, A. 1950 ComputingMachineryandIntelligence Mind , Vol. 59, pp. 433- 60. " Visual Routines in Visual " Ullman, S. 1985 . , , , , Cognition ed. S. Pinker MIT Press . , MA , pp. 97- 159 Cambridge U.S. Anny . 1986 Field Manual No. 7- 7J. Department the Army, WashingtonDC. . of , VershureP., Krose B., and Pfeifer, R. 1992 " DistributedAdaptiveControl: The SelfOrganization . , , of StructuredBehavior" Roboticsand Autonomous , , SystemsVol. 9, pp. - 96. 181 Vershure P., Wray, J., Sprong 0 ., Tononi, G., and Edelman G. 1995 " Multilevel . , , , " Analysis of ClassicalConditioningin a BehavingReal World Artifact, Roboticsand Autonomous , , SystemsVol. 16, 1995 pp. 247 65. Wallace R. 1987 " Robot RoadFollowing by AdaptiveColor Classificationand Shape . , " on , Tracking Proceedings theIEEE InternationalConference RoboticsandAutomation of --63. , Raleigh NC, pp. 258 , Wallace R., Matsuzaki K., Crisman J., Ooto, Y., Webb J., and Kanade T. 1986 . , , , , , " " in Progress Robot Road Following, Proceedings the IEEE International Conference of on Roboticsand Automation SanFrancisco CA, pp. 1615 21. , , Wallace R., Stentz A., Thorpe C., Moravec H., Whittaker W. and Kanade T. 1985 . , , , , , , " First Resultsin Robot Road -Following," International Joint Conference Artificial on , Intelligence(IJCAI-85), Vol. 2, Los Angeles CA, pp. 1089 95. Walter W. G. 1953(reprinted1963 . TheLiving Brain, Norton, New York. , ) J. 1994 " On Sign-Board Based Inter-Robot Communicationin Distributed . , Wang " Robotic Systems Proceedings the IEEE International Conference Roboticsand on , of Automation pp. 1045 50. , ." PrimitivesSupporting Traffic Regulation Controlof Mobile and , Wang J. 1995 Operating " RobotsunderDistributedRoboticSystems Proceedings theIEEE International , of on , , , , Conference Roboticsand Automation Nagoya Japan June pp. 1613 18. Wasserman 1989 Neural Computing Theoryand Practice, VanNostrandReinhold . : , P. , New York. WatermanT. 1989 Animal Navigation ScientificAmericanBooks New York. . , , , Watkins C., andDayan P. 1992 " Q-Learning" MachineLearning, Vol. 8, pp. 279- 92. . , , ,

475

References ' Watson J. B. 1925 Behaviorism Peoples InstitutePublishingCo., New York. . , , Waxman A. M., LeMoigne J., and Srinivasan B. 1985 " Visual Navigationof Roadways . , , , " on , Proceedings the IEEE InternationalConference RoboticsandAutomation , of St. Louis, MO , pp. 862 67. Waxman A ., LeMoigne J., Davis, L., Srinivasan B., Kushner T., Liang, E. and , , , , " ." Land Vehicles , , SiddalingaiahT. 1987 A Visual NavigationSystemfor Autonomous IEEE Journal of Roboticsand Automation Vol. RA-3, No. 2, April , pp. 124 41. , 's Webster Ninth New CollegiateDictionary, 1984 Merriam-WebsterSpring . , field , MA . Weiner N. 1948 Cybernetics or Control and Communication Animals and Machines . in , , , Wiley, New York. ." Model for DistributedHeterogeneous Weitzenfeld A. 1993 A HierarchicalComputational , " Technical , , Systems ReportTR-93-02, Centerfor Neural Engineering University of Southern . California Los Angeles , " , Werbos P. 1995 " BackpropagationBasicsandNew Developments in TheHandbook . : , , Brain Theoryand Neural Networks ed. M. Arbib, MIT Press Cambridge MA , pp. , , , of 134 39. " Werner G., andDyer, M. 1990 " Evolutionof Communication Artificial Organisms . in , , Technical UCLA-AI -90-06, AI Laboratory University of California Los Ange, , Report les. Wilkes, D., andTsotsos J. 1994 " Integrationof Camera . Motion Behaviorsfor Active , " on , , , Object Recognition Proceedings the IEEE Workshop VisualBehaviors Seattle of WA, June pp. 10 19. , Wilson, E. O. 1975 Sociobiology TheNewSynthesisBelknapPressCambridgeMA . . : , , , " " Winston P 1975 LearningStructuralDescriptions . from Examples in Psychology , . , of , , , , ComputerVision ed. P. Winston MIT Press Cambridge MA , pp. 157 209. Woodfill, J., and Zabih, R. 1991 " Using Motion Vision for a Simple Robotic Task" . , on , , WorkingNotes 1991AAAI Fall Symposium Sensory Aspects RoboticIntelligence of Asilomar CA , November pp. 162 65. , , " . YamamotoM. 1993 " Sozzy A HomloneDriven Autonomous : VacuumCleaner Proceedings , , - 23. of Mobile RobotsVIII , pp. 211 . Yamauchi B. 1990 " BehavioralMemory Techniques Robot Navigation" unpublished for , , Laboratories Malibu, , Artificial Intelligence Center HughesResearch , , report CA. Yamauchi B., and NelsonY . 1991 "A BehaviorBasedArchitecturefor RobotsUsing . , Real Time Vision," Proceedings the International Conference Roboticsand on of Automation Sacramento , pp. 1822 27. , , CA Yanco H., andStein L . 1993 "An AdaptiveCommunication . Protocolfor Cooperating , , Mobile Robots" From Animals to Animats Proceedings the SecondInternational : , of on / Conference theSimulationof AdaptiveBehavior, Honolulu, In , Mit Press Bradford Books Cambridge MA , pp. 478- 85. , ,

476

References ." Yoshida E., Yamamoto M., Araj, T., Ota J., and Kurabayashi D. 1995 A Design , , , , " Medtodof Local Communication Areain Multiple Mobile RobotSystem Proceedings , on , , of the IEEE International Conference RoboticsandAutomation June pp. 2567 72. " " Workshop . Behaviorof Multiple Autonomous Mobile Robots Yuta S. 1993 Cooperative , , in on Needsfor Research Cooperating Robots IEEE International Conference on , Roboticsand Automation" Atlanta GA. , , Zadeh L. 1973 " Outline of a New Approachto dIe Analysisof ComplexSystems . and , Decision Process ," IEEE Transactions SystemsMan and Cybernetics Vol. 3, es on , , , No. 1, Januarypp. 28- 44. , ." , , , , , Zapata R., Perrier M., Lepinay P., Thompson P., and JouvencalB. 1991 FastMobile " -Structured Robotsin ill Environments Proceedings theI EEFl R JInternational S , of ' on , , ( Conference Intelligent Roboticsand SystemsIROS 91), Osaka Japan pp. 793- 98. ." , , Zelinsky A ., Kuniyoshi Y. 1996 Learningto CoordinateBehaviorsfor Robot Navigation " Robotics Vol. 10, No. 2, pp. 143 59. , Advanced , . " , A., Kuniyoshi, Y., Suehiro T., and Tsukune H. 1995 Using an Augmentable , , Zelinsky " Resource Robustly and Purpose to , fully Navigatea Robot Proceedings of the IEEE International Conference Roboticsand Automation Nagoya Japan pp. on , , , 2586 92. : Zeltzer D., and Johnson M . 1991 " Motor Planning An Architecturefor Specifying . , , andControllingthe Behaviorof Virtual Actors," Journalof Visualization Computer and Animation Vol. 2, pp. 74- 80. , ." Color Projectionfor RobotRoadFollowing , Zeng, N., andCrisman J. 1995 Categorical " IEEE International on , , , , Conference Roboticsand Automation Nagoya Japan pp. 1080 85. -

Name Index

, A. - , Agah , 381382399 , . , , , , , AgreP, 73 169214258260272 . Albus , 21 22 423 , J. - , Ali, K., 378415 , Anderson, 171 , T. ArbibM., 40 42 , 69 82 86 118141 , , -43 , , , , , 143254286 , , ArkinR., 28 69 82 84 142143155214 , , , , , - , , , 283377381383409412432 , , , , , , ,W , Ashby ., 8, 133432 Badier , 171 , No , Ro , , Bajcsy , 202270280 Bakker , 400 , Po Balch , 129155161377408 , To , , , , Ballard , 271272 , Do , BarberAo 166 a , , Bartlett , 42 , Fo Beer , 57 69 , Ro , , Go , - , Bekey , 172381382399 Bellman , 423426 , Ro , Biederman276 , 10 ' Birnbaum, 278 , Lo Blake , 270 , Ao , Lo Bogoni , 202 Bohm , 127 , Co Bonasso 88 234 , Po , , Borenstein 189 , Jo , Bosser , 106125172 , Uo , , - , ,K Bowyer. , 201202298 , Mo Brady , 2 , Vo - , , , , , Braitenberg 10 14 133325342 427428 Brand , 278 , Mo Brill, Fo273 , , Do Brogan , 170 Brooks , 1528 74 82 106131134166 , Ro , , , , , - , , 168170298313424 , , , , Budenske202 , J., Bunke , 298 , H.

Cameron 217218 , J., CaoY., 369 , Chalmers, 425 , D.

Chapman73 169 . 258 . 272 . D.. . . 214 . 260

Chatila , 232 , R. ChielH., 57 69 , , Christensen298 , H., Connell , 77 98 167229 , 316 , J. , , , - 230 -410 CookD., 409 , ,P Cooper., 278 Crisman, 289 , J. , J. Crowley, 269 Culhane, 276277 , S. Dean , 125 , T. , J., 53 Deneubourg Dennen , 181425 , D. , Dickmanns 288 , E., Donald , 263 , B. Donath , 171 , M. Drexler , 39 , E. , DyerM., 380 Eklundh ., 270298 , J.-O , EItes , 189216 , A. , & penscheid, 58 , K. Ferrell , 77 129 , C. , Ferrier , 298 , N. FlynnA., 27 439 . , - , , , , , FirbyJo74 169219 Franklin , 381 , Ro Franceshini 55 56 , No , Fu Do278 , , , D. Gage , 394 Galanter, 169 , E. Gallistel , 184 , C. Gardner , 244 , H. GatE., 82 218 , ,

478

Name Index

. Gaussier 325 , P, , M. Georgeff , 169 Gerald , 430 , M. Gibson J., 46 51 244 , 255 , J. , , -245 Gini M., 202 , GoelA., 217 , Goss , 53 130 , S. , , R. Gropen , 171
Hammond , 278 , Ko Harmon , 381 , Lo -Rodi , , , Hayes , Bo 125 129231 HebbDo321 , , Henderson , 171 , To Hendricks , 222 223 , Ao Hillis, Do 168 , , Jo Hodgins , 170 Holldobler , 52, 364 , 383 -365 , Bo Horswill lo 129 133264 298 , , , , ,

House , 40 , Do HsuJo391 , , Hull Co180 , ,

MacLennan 381 , Bo , Mahadevan316 , So , Maes , 112167168170171313 , Po , - , - , Malcolm , 181212231 , Co , , MarrDo254256 , , , Martial , 199 , Ho Mataric , 125129136393395396 , Mo , , , , , Ao Meystel , 210 MilianJo327 , , MillerDo172 , , MillerGo169 , , , Mo - , , , , , , Minsky , 14 1544 168254330376 - , 427428441 MintzMo280 , , Mitchell , 232 , To Mittelstaedt, 242 , Ho Moravec , 18 19 106189216426 , , Ho - , , , , -428 439441442 , , , Ro - , Murphy , 217218286288 Neisser , 42 , 207244246 , Uo -43 , Nelson , 297 , Ro Nilsson , 96 118166306 , No , , , Noreils , 232 , Fo Norman , 42 207209 , Do , Pahlavan, 270298 , K. , Pau , 284 , L. Pavlov, 322 , I. , D. , , , , , , , Payton , 75 112114129170195199 273 Penrose, 128424 , R. , Pfeifer , 323 , R. , J. -43 Piaget , 42 , 310 PinF, 347 , . Pomerleau 291 , D., Premwti ., 362368 ,S , Pribram , 169 , K. . , P, 295 Prokopowica , R. Quinn , 58 Ram , 217349 , A. , Reece , 260 , D. , C. Reynolds, 170

1beraTo286 Il , , , Go Jacopini , 127

, J. Jennings , 263 Johnson , 166 , S.

, L. -88 , Kaelbling , 87 , 118166 Kahn ., 171 ,P Kahn , 295 , R. KantI., 41 , Khatib , 98 143 , 0. , , KlugeK., 286 , S. Koenig , 197 , K. Komoriya , 82 , K. Konolige , lIS, 346 Koren , 189 , Y. Kosecka 280 , J., , B. , Krogh , 98 234 Krose , 323 , B. Kube , 374 , R. , Y. , , , Kuniyosbi , 200277393400
Latombe -C., 98 , J. Lim, W., 172 Lin, F., 391 LorenzK., 47 48 ,

, D. -87 , , 212 -223 Lyons , 86 , 118142 , 222 , 286

, J. , McCarthy, 14426 McFarland, 51 106172 , D. , , MacKenzie, 82 376412 , D. , ,

Rosenblatt , 44 , Fo Rosenblatt , 112 113 170 , Jo - , Rosenscbein, 87 88 166 , So - , , , RuspiniEo 115

Saffiotti , 115230346 , , , A. Saridis , 24 234 , G. , Schank , 305 , R. . Schneider, 207 ,W , M. Scboppers, 74

479

NameIndex

Scutt , 325 , T. Searle, 425 , J. Shafer., 260 ,S shallice , 207209 , T. S herring , C., 180 ton Shiffrin , 207 , R. Simmons, 197 , R. SiID, K., 339 S Skinner F, 45 180207 , B. . , , Slack , 100169221 , M. , , Smithers, 181212231 , , , T. Stark , Lawrence , 277 , 203 Stark , Louise - 202 , 201 Steels , 106178338 , , , L. Stein , 298382429 , L. , , Stentz , 199 , A. Stone , 125 , H. , E. 217 .Stroulia,,278295 Swain , M. , Tachi ., 82 ,S Takeuchi, 439 , S. Tanimoto, 178 , S. , C. Thorpe , 286 TbnmS., 355 , , N. -48 Tmbergen, 47 , 363 Tsotsos, 28 240270276277 , J. , , , Thrban , 178 , E.

Zabih , 295 , R. Zadeh , 170 , L. , A. Zapp , 288 , A. Zelinsky , 200 Zeltzer , 170 , D. , H. Zhang , 374 Zrehen , 325 , S.

Uh , T., 270 Iin Ullman., 169 -258 , S , 256 , 260 . Vershure , P, 323 Viola ., 98 ,P
Wallace , 289 , R. Walter . Grey 10 133 ,W , 8- , , J. Wang , 385 Watanabe 341 , Y., Webber , 111 , B. Weitzenfeld, 255 , A. Wellman , 125 , M. Werner , 380 , G. Wiener , 8 , N. Wilkes , 210 , D. Wilson 52 364365383 . E.. . - .
. mston ., 320 W ,P WoodfillJ., 295 ,

Yamauchi, 188200297 , Bo , , Yanco , 382400 , Ho , Yoshida , 381 , Eo YumSo362368 , , ,

Subject Index

A* , 197 199215 217 Arbitration Coordination . See , , , , behavioral , Abstract schema arbitration , 255 . Abstraction -language143 166 231 ARCH , 117 118 135 , , , , , . 320 program 233 414 Architecture , -partitioned -selection - 110112149 Abstraction evaluator action , 233 , 109 , , , - , ABSTRIPS 167168174 , 14 ACBARR349 351 , , 231 agent Accommodation ALLIANCE - 375420 , 310 , 372 , animate , 169174 , 372 , Acquiescence - 373 398 agent , Actionperception , 245 , 273 ARC172 , cycle -246 Activation Atlantis , 218222235 , 214 - , levels141 149 167 AuRA130214218229234235 , , , , , - , , BART171 , 167 , spreading values324 . circuit87 , 166174 , -89 , , 167 , , (see ) Adaptation alsoLearning colony - 168174317 DAMN112113169170174199401 , 306 , - , - , , , evolutionary - , , , -339 on line 217 225 338 , 350 definitions , 125 as , deliberative Jhierarchical24 , 21 planning , 214 224 criticelement reaction , 326 , 172 Adaptive dynamic heuristic critic(AHC . See , criteria ) Adaptive Learning evaluation . 128129 heuristic critic , 126 adaptive expressiveness128 , , robot Advising214 222 generic , 232 Afferent , , 28 , - , , inputs36 hybrid , 117206207346429 Affordances 91 201 202 244 245 255 . 46. . . - . . L-ALLIANCE - 399 , 398

299 AFSM132137 , See , Agent Agent Software Alder328 , ALECSYS - 338 , 337 ALFA219 , Allen 133140 , , Allothetic , 242

ALV See . DARPA autonomous vehicle land program ALVINN 291 293 401 , , , Anchovies , 154 Animation , computer - 171 361 , 170 , Ant, 52 54, 130 364 , -365 , Antibody438 , Antigen438

NASREM , 124126 , 22 , -based , niche , 52172 , 126 organizing principles , 129130 -reactor , 222226235 , 214 - , planner reactive deliberation - 234 , 233 scbemas , 141165174 , motor - , SFX286288 , SIVS258262 , , skillnetwork - 171174 , 170 , SiDarty , 172 Cat SSS , 229 , 109 , , - , , subsumption , 112126130141164 166167174313370372435 - , , , , ,
, 233 supervenience teleoreactive , 233 agent , 381 tropism system cognitive , 399 401

482

SubjectIndex Architecture .) (cont UGVDemo , 401 403 D Ariel, 4 -379 , ArmyAnt project378 ArtificialInsect , project57 Artificialintelligence " (AI) distributed , 142 360 393 , IS , , , origins 14 15 , fighting51 , , fly-at-a-window165 183 - , , , , , - , gain 93, 108 113 114 120 142 149 150 326 , 129 , , granularity , 136 142 166 , , , boming185242 370 , , layers132 136 137 letting, 367 libraries143 163 164 171 , 424 , , , , strong , 426 Artificial 361 life. , 92-93 mapping . See , assemblage , Assemblage Behavior mating51 micro, 72 , 206 -225 Assembly , 223 , 231 Assimilation , 310 network142 , Attention , 270 object recognition definition24 , , 230 preplanned focus , 171 240 251 273 276 279 of - , , , , , , , primitives116 142 288 reflex8, 26 327 , , in humans - 277 road , 276 (driving 256 286 ), , following - 208 resources , robot 57 65 , , requiring207 visual258 social363 364 428 , , - , Attila 140 , , 374 stagnation , 376 AuRA(Autonomous Robot ArchitectureSee -297(See alsoVision , , ). tracking296 Architecture , AuRA , tracking ) computer AUSS6 traits 419 , , Autonomouse visual298 , 337 , Awareness , 207 willed 207 , 166 , -based Behavior systems Baboon , 362 , 26-27 aspects Backpropagation , 322 355 356 , 44 , , -79 , design69 , 141 142 172 173 Bat 34 185243 , , , dogma131 , Bee 60, 185 362 425 features , , , , l24 Behavior , 163 reconfiguration animal21 22 39 41 47 52 58 62 154 nIle -based -97 120 313 370 372 , - , - , - , , , , , 95 , , , 365 366 use fulness127 128 , - , , , 48 , - , , , , Language , 133 134 137 140 , 97 , assemblage , 87 116 118 120 130 309 Behavior 172 326 376 , Belief 227 , 104 , , assembling - 119370 autochthonous Birds 154 185 365 367 , 171 , , , , Bolds 170 automatic , 207 , chase Brachiation , 337 , primate60 , Brain classification , 77 79 , robot cortex36 185 243 continuous functional , 98- , , , , auditory encoding 104 function36 142 120 143 , , , , 9, 185 cybernetic 10 hippo campus , 194 definition24 , , 36 hypothalamus , 431 discrete limbicsystem - 36 427 , 93 , , , , 35 , encoding -98, 120 133 166 168 170 171 . cortex43 , parietal structure -37 , 35 , , display50 364 visual areas -36 184243 244 , , , 35 , , docking279 361 dnmken , 165 sailor Brown , 265 University BUSTER , 278 , 24 , - , , emergent , 27 105 107 120 167 Buzz 143 165 , 89 , , , encoding - 103 130 , 51 escape C Language , 229 , 81 experts , 328 , 79 expression - 89 Cairngorm

483

SubjectIndex -MellonUniversity -20 ) (CMU , 18 , Carnegie 169 198 199232 286 289 291 , - , , , , Case -based . See , reasoning Learning -based case Case Western Reserve , 57 University Cat 243 , CCDcamera , 278 , 250 CEBOT361 363 , Central nervous ( ), systemCNS 33 41 (See alsoBrain ) Grid Certainty , 189 190 Chemotaxis -54 , 53 Chinese roomargument - 426 , 425 , Chip 169 Chromosome . 331 -Thring Cburch thesis424 , Classifier , 337 , systems - 338 356 CMURover18 , -20 CNRS55 , Cockroach , 56 60 69 440 , 34 - , , , , -299 Cog 140 298 , 302 303 models , 42 Cognitive failure219 , Cognizant rnmmllnicat1nn animal50 60 62, 362 364 366 , , , broadcast , 432 , 392 content - 385 , 382 evolution , 380 381 of , 60 honeybee - 61 , implicit 385 robotics interrobotSee , ( Multiagent communication ) , minimizing131 , range381 382 , 391 signboard state384 -387 , , 135 subsumption , 74 Competences, 167 168 , environmental , 51 Competition , 240 , computational , 263 Complexity , 322 , , classical - 325 356 Conditioning resolution Conflict , 94 . See networks Connectionism Neural Consciousness , 424 427 , 422 . See , consuming Consuming Navigation Control , 307 adaptive automatic - 209 , 207 -loop closed , 242 feedback , 431 434 ,8 , , , - , fuzzy 115 342 349 356 hierarchical Deliberative ) (See systems -loop open , 242 reactive , 24 threads -209 , 208 willed 207 20S , , 316 , Convergence , 321 333 , 393 Cooperation observation by Coordination , behavioral arbitration , 74, 111 113 133 139 140 , 10 , , , - , 142 166 16S170 197 376 , , , , , , lOS - , , , competitive , 111 113 129 133 16S , lOS - , , cooperative , 113 115 129 142 143 functions , lOS309 412 , Sl , , fusion 113 116 142 , - , maximum selection , 50 -based - , , schema , 149 155326 377 , 109 subsumption , 135 ,' , voting l12- 114 169 170 s , Coulomb law 98 Credit , , assignment problem309 311 Critic 311 329 337 , , , Crossover , 339 , 332 K2A , Cybermotion robot 172 , 8- , Cybernetics 14 267 D* , 199 , 204 -200 DAI. See Artificialintelligence , distributed DARPA autonomous vehicle land (ALV ) , , program195275 DARPA UGVDemo program170 229 n , , , 401 403 410 420 - , Deadlock , 391 392 , 385 Dead , 247 reckoning - 248 Decision , theory409 410 Defense , Mapping Agency195 Deformation , 102 103 zones of ( ), Degree freedom DOF 90 Deliberative systems characteristics , 21 , 209 examples use knowledge , 206 , deliberative ) (See planners Planning Delta , 321 322 nile -Shafer methods , 217 Dempster - , , robot , , Denning , 165 217 378 379 384 418 419 of , 249 Deparbnent Defense Design /constrained -72, , 69 ethologically guided 120 143 168 171 , , , driven -79 , , experimentally , 74 , 120 133 136 137 168 170 171 - , , -based motor schema , 155 156 situated , , , 72 activitybased - 74, 120 166 169 , 136 subsumption Desires , 227

, -387 goal384 guaranteeing- 392 - , 385

484

SubjectIndex , 90 Dimensionality Discrete event systems , 284 ~ , 280 -28 . See behavior Behavior , display Display Distinctive , , places192 204 , , Dog 322 331 . DOFSee of Degree freedom Drexel , 22 University - 23 , 232 Dynamical systems approach - 233 , 89 Dynamics -90 Echolocation , 243 Ecological ,I approach dynamics , 27 niche 27 51 52, 125 174 211 217 , , , , , , 334 337 339 442 - , , , 128 pressure robotics1 , R2 , Edinburgh , 328 330 Efferent , signals36 Electrotechnical ( ), LaboratoryETL 393 Embodiment , 135 , 26 . See , emergent Emergence Behavior Emotions Robotemotions . See , Endocrine , 430 system - 431 , 431 Energy management Environmental Research Institute Michiga of E Frame reference of , 256 egocenbic , 271 -centered , 271 object - , -72 , , Frogtoad 34, 38 41 69 , 243 247 / FSAsSee . Finitestate acceptor diagrams Fuddbot172 , Functional notation80 81 120 , - , GA-Robot333 , GaitsSee . , Walkinggaits Ganesha 1 , 40 - , , 87 , , , Gapps - 89, 95 96 120 164 166 234 General visionproblem268 , Genetic , 155 , - , , algorithms , 217 331 342 356 400 , 76 , - , - , Genghis , 133 136 137313 316 374 , 143 George , 164 165 Tech , , , , Georgia , 143349 377 383 412 Gerbil 185 , Gibbon60 61 ',s Goedel Incompleteness Theorem , 424 GPS Global Position Sensor ( , ). System See GPS Gradient descent , 331 , 322 Gradient . 195 196 field -

( ERIM381 ), , 184 Epistemology , 440 Equivalence - 442 Ernie Bert382383 and , , 32 - , Ethology , 47 52 360 Evolution , 51 , 46 , , , , - , , Expectations240246251272274275 278 , 241 Exteroception- 242 Fault tolerance, 77 360420438 , 75 , , , FERMI , 286 Finite acceptor ) diagrams-86 state , 81 , (FSA 118120131157169194217226 , , , , , , , , 280281384 , 391392 - , -385 Fish154362364 , , , Fitness function - 334342356 , 332 , , Fixation - 273 , 272 -action Fixed , 48 , , , patterns , 51 57 94 268 , 166 - , Flakey , 227228346 . See , flocking Flocking Navigation , , 55 Fly house Focus attention Attention of of . See , focus . See , foraging Foraging Navigation Forced relaxation , 224 assumption Formal Methods , 84 Formations Navigation . See , fonnation Fovea , 276277 , 203 ,

. See , Hexapod Robothexapod Hierarchical controlSee . Deliberative systems Hn.ARE 17 18 232 . , - , Hill climbing 192 322 , , HMMWV229 401 403 , , Holonomic , 90 Homeostasis , 306 430 432 438 , 216 , - , Homeostat - 433 , 432 Homeostatic control215 , 443 , -216 Hormones - 432 435 437 , 430 , transform , 279 Hough ArtificialIntelligence Center114 , Hughes /reactive ) systems Hybrid(deliberative evidence - 209 , 207 biological , 212 components

Habituation - 307 , 306 HACKER , 14 distance -319 , 318 Hamming HARV , 164 , 143 Harvard University , 300 , 298 , 15 HearsayD HEGI060 ,5 , autonomous , 251 Helicopter Herbert , 168 , 78 Hero KX232 2< > ,

485

SubjectIndex connections , 212 , 209 interface , 214 strategies Lateral inhibition35 112 , , Lawof EtIect 310 311 , Lawof Universal Gravitation , 98 ( also ) Learningsee Adaptation

, 213 layers , 218 . 212 synthesis

mM , 229 316 , Ideothetic , 242 network438 , Idiotype . See , Imagination Robotimagination Immune , 438 systems , 165 - , Impatience , 372 373 398 -functional Indexical , 73 Infrared sensor Sensor . See , infrared , 309 Indexing problem INS(Inertial Sensor , ). navigation system See INS Insects185 362 , , , 131 Intelligence Intentions, 24, 43, 46, 142 167 182 201 , , , , ) 227 360 ,

heuristic (AHC 312 , critic ), - 313 adaptive 325 . 356 -329

10Callisto , , Ganymede - 165 , 163 IRM(Innate mechanisms, 63 ), 48 releasing ISRobotics - 138 , 137

Jet (JPL Propulsion Laboratory ), 218 School Medicine of Johns Hopkins , 40

Khepera - 343 , 342 Kin recognition , 247 393 4 ) , 51 , , (1 Kinematics -90, 248 339 340 , 89 , Knowledge , , , , 66 avoiding representational, 131 211 263 268 basis179 , characteristics - 184 203 , 179 , conb ' Oversy , 178 definitions , 178 , 212 necessity -to need -know 185 , , 272 perceptual - 273 persistent , 182 183 190 203 , 179 - , ,

, 15 , strong , 173224 , 181 taxonomy time horizon , 184 tradeoffs , 180 , 182 , , transitory - 183186203 , 182 types weak173224 , , KTH 298301 , ,

batch310 , case -based , 308 349 , 217 , -354 communication - 383 , 382 continuous , 310 deductive , 310 definitions , 306 dimensions , 310 , 308 evolutionary , 333 337 -based -355 explanation , 350 , fuzzyrules 347 349 Hebbian , 323 325 , 321 , /neural340 342 , , hybridgenetic imitation395 400 401 , , inductive , 310 , lazy 350 machine , 218 -based , 350 , 308 memory momentum , 350 , 308 multistrategy , 37 neurobiological for opportunities , 308 310 , 309 problems -313 , -320 , , , Q , 312 , 316 318 , 355 356 396 399 438 , reinforcement , 310 320 356 396 , , 307 - , , -398 438 social395 401 , , 311 , supervised , 330 347 , 311 unsupervised Least commitment , 216 217 235 , 214 - , . Behavior , lekking LekkingSee LIFIA, 347 USP 81 87 97 219 , , , , Localminima99, 151 , Localization - 181 187 395 , 180 , , Locus superior of command , 50 Loebner , 423 competition Logic formal 166 , , , - , fuzzy 115 342 343 345-346 multivalued , 230 231 , 115 sensors /actuators ( LSAs 202 ), Logical LTM. See , Memorylongterm Manifolds102 103 , Map a priori 186 194 , , -200 - . -204 , , cognitive181 184 186 203 , 229 geometric

LAAS 17 , Landmarks , 193264 325 , 184 , , LAURON328 , -329 Laser scanner Sensor . See scanner , laser Lateral nucleus , 243 geniculate

486

SubjectIndex . Map (cont) instantaneous obstacle ), 189 (10M meadow , 215 , perilsin using 195 , 200 purposive Qualitative - 193 , 192 sonar216 , , , , spatial180 186 193 tectal 185 , Mapping behavioral , 92-93 , 184 , retinotopic , 243 258 as , sensing , 263 269 , 36 , somatotopic , 184243 , 244 tonotopic , 36 topographic , 56 MARGE346 , -348 Markov models197 , Mars /Lunar Rover77 172 221 , , , Massachusetts Institute Technology ) , of ( MIT II , 38 130 133 141 168 264 298 392 , , , , , , , , 429 439 , MAVERIC412 , Maximum , 50 selecting system MELDOG82 , function343 344 356 , - , Membership Memory action188 189 , associative , 328 330 356 , 183 , , behavioral - 190204 . 186 . , forgetting183 , global22 , . . - . , longterm 37 40, 183 185 186 204 310 shortterm 37 40, 155 183 185 187 204 . , . , , . . 216 217 310 - , wall, 188 Metabolism . 431 Meta , 429 Toto of , MilitaryUniversity Munich288 , 242 Millipedes Mission . 377 410 415 430 Lab , - , MITI. 428 Mobilemanipulator , 144 153 165 168 . 77 . , . . See , modularity Modularity Software Molecular , 439 manufacturing . . 243 Monkeymacaque MSSMP .6 robotics Multiagent . 359 advantages - 360 , 394 applications , 376 assemblage communication - 51 360 364 366 369 - , , . 50 . . 372 378 . 443 , -392 . 367 congregation methods -395 . 394 coverage
, designdimensions365 368 , 360 disadvantages emotionsin, 428 , edtologicalconsiderations362 364 , heterogeneity365,369 immunesystems , 438 in interference360 , metrics 367 368 , micro-, 439, 441 missionspecification41(}..415 , , system 137 teamsize 369 , , teleautonomy377, 417- 420 Mutation, 332- 333 Mutual exclusion 385 ,

, 60 Nagoya University , 428 -440 , 438 Nanotechnology NASA22 219 , , NASREM Architecture . See , NASREM National Institute Standards Technology of and (NIST 21 23 ), National Taiwan , 391 University
. NATsSee Navigational templates Naval Research Laboratories , 338 Navigation animal40- 41 184 185242 362 364 , , - , , classroom , SO82 84, 87 109 111 , 67 , , , 361 consuming , 385 , 202 doorway , , , flocking361 363 371 , - , - , , , , foraging130 131 137 139 157 161 163 185 361 364 366 372 381 384 , , , , , , -386 fonnations , 377 401 403 410 , 361 , , , grazing361 , homing ( Behavior ) homingSee indoor165218 229 265 327 , , , , , office 16 17 194 197 227 229 231 , - , , , , , outdoor165 195 199 , 218 , , , -200 , 192 , qualitative - 193204 robot 56 152 165 194 , , , , -based , - , , schema , 69 143 161 333 395 three -dimensional , 152 , 144 , 100 , Navigational templates , 221 235 Navlab 289 291 294 (I), , , D Navlab , 199 Navlab , 293 295 5 , NerdHerd 370 372 420 , - , Neural circuits34 393 , , model57 , networks - 44 47, 183 184 307 - , , 43 , , 320 , 356 -331 Neuron32 33 , -Pitts McCulloch model43 ,

487

SubjectIndex Neuroscience - 44 , 32 Newtermproblem309 , NewYorkUniversity Medical Center , 40 NOAH 14 223 , , Across NoHands America293 , Noise99, 151 155 157 163 165 , , - , , NOMAD 324 , Nomadic - , , 126 , Technologies , 200 249 250 327 338 , Nondetenninism , 381 , 107 Nonholonomic , 326 , 90 NorthCarolina State , 346 University Northwestern , 278 University elaboration -227 235 , 226 , as , expression FSA 217 , global209 internalized - 196203 273 , 195 , , , 393 recognition , 74 sketchy Universal , 74 Planning , 223 anytime , , avoiding15 67 deliberative -212 , 209 hierarchical -212 , 209 in hybridsystems , 214 mission215 . -216 motion360 , multisb , 217 ' ategy , , , path 98 231 234 route215 216 , , 21 scope Policy312 , , , , Polly 133 140 264 266 Portautomata , 86

, , Opticflow 55 255 Osaka , 319 University Pandemonium 15 Pathfinder , 219 Path , 242 integration , -74 , , Pengi73 , 129 258 272 , , 264 , , Percept -265 268 272 287 288 Perception -oriented143 151 201 215 240 action , , , , , , 245 246 265 , 279 299 - , -270 , active202 240 265 269 272 299 , , , , - , ascommunication -247 , 246 distributed , 392 396 409 410 420 - , , 360 - , and ethology , 246 247 , 254 generalized modular - 265 , 254 -to-know 143 201 240 need , , , neuroscience , 242 244 and and -246 psychology , 244 , 286 - , , ron Percept , 44, 321 322 324 330 Perceptual classes , 262 264 393 , 240 - , , 263 equivalence , 150 performance schema Schema , perceptual ) (See , 268 , , -283 sequencing - 269 273 279 , 299 , 240 specialization , , , trigger92 93, 209 217 280 Pheromone , 53 Phony , 84 Pony Phototaxis , 337 Plan definition14 , , digitalterrain 195

Potential . 436998 . 113 . 120 fields . . -99 . 151 .

172 190 232 234 325 . . . . Primate , 364 , 243 Procedural , (PRS Reasoning System ), 214 226 230 235 - , - 242 248 , 241 , Proprioception , 393 Prosopagnosia Prosthetics , 172
, Psychology45- 47 behaviorism45, 180 181 207 , , , , cognitive 46, 181 203, 207 209, 267 definition, 32 , ecological 46, 51, 245 , gestalt 44 , gibsonian 244- 245 fusion, 286 modelfor sensor , 45 sensory psychophysics

. See .Q Q learning Learning . 211 Qualification problem . See , Qualitative navigation Navigation qualitative of . Qualities sociality365 . 425 Quantum computers

R-l robot138 , R-2robot374375 , R-3 robot394 , RALPH , 293

Rat 185 194 , , Randomness , 155 , 401 Ranger

RAPs . 77 129 . 219 -235 -. 74 . . 169 . 234 . . .

488

SubjectIndex RCS22, 166 , Reactive action . See packages RAPs Reactive systems , 206 assumptions characteristics- 67 , 66 definitions , 66 , 24 Recruitment -384 , 383 Reflex , 47(See Behavior es also , reflex ) RenandStimpy143 164 165 , , Rensselaer Institute RPI 24 ( ), Polytechnic (see ) Representation alsoWorldmodel deictic272 , -273 drawbacks , 203 -based - 202 204 function , 201 , , 189 , geometric , 201 203 , , , , , grid 187 189 195 199204 234 in genetic , 331 algorithms mental244 , metric 179 182 197 204 , , , , , 200 perceptual - 203 , 192 , qualitative - 193204 relational18 179 182 , , , roadmodel289 290 , , , , , spatial182 186 193 199 , 192 , topological , 197325 , 332 , -364 Reproduction - 334 363 Resource distribution -367 , 366 Response , 93 categories continuous , 98 - , fight or flight 432 , , global 108318 instantaneous , 90 orientation , 93 144 , 89 , , range90-91 (magnitude89 144 ), , strength unconditioned , 324 , 322 Reuse Software . See , reuse Rex 87 166219 234 , , , , Road . Robots , roadfollowing followingSee , Robby221 Roboroach - 440 , 439 Robot aerial218 394 , , ann 231 241 270 297 , , , , bio -robots52 62 , biped2-3 , boxpushing - 319 374 376 377 , 316 , , cellrepair439 , cellular361 , cockroach -59 439 440 , 58 , - , , 84 , , competitions , 86 118 119 165 217 218 251 346 348 393 , , , , control , 20 , Spectl1lm , 67 207 , 394 convoying definitions , 2 ,1 , delivery231 , docking279 283 education , 439 emotions , 11 427 428 443 in , - , , 134 -327 exploration , 326 , 439 ordnance , 439 explosive disposal , gnat439 , 171 , , , grasping /manipulation , 225 231 337 355 hand225 , hazardous cleanup waste , 374 head298 303 , - , , 4-5 -59 - , , hexapod , 58 , 75 77 136 137 140 313 , 328 329 342 438 -316 - , , , 428 imagination - 430 industrial , 241 , , juggling 297 350 landmine detection , 394 LEGO 11 , micro, 439 441 , , military 218 247 mind 422 430 , nano, 438 439 nonholonomic - 91 , 90 omnidirectional , 91 , -380 palletmoving378 ,4 quadruped reconnaissance , 401 409 410 , 394 , roadfollowing driving 44, 256 260 , -262 , ( ), 275 286 288 295 , , -based - , , schema , 163 165 171 395 selfassembling - 440 , 439 , 319 shooting - 320 , 140 subsumption - 141 surveil , 125231 394 401 lance , , , taxonomies - 370 , 368 teamsSee robotics ( Multiagent ) tourguide 140264 -266 , , , 218 - , vacuuming , 428 429 435 437 wheelchair , 168 172 , 98 , Robotics , origins 15 20 rehabilitative , 172 Robustness , 131 163 166 , 129 , , Robuter , 326 Rockettes , 439

m Rocky , 172 IV Rocky , 220 , 342 Rodney Router , 217 RS(Robot Schema , 86 88, 118 120 , , ) model 223 , 255 -224 Rules

behavioral. 139 . 313 . 370 . 137 . 232 -315 - 371

489

SubjectIndex classifier - 338 , 337 -344 - , , fuzzy 343 , 346 349 356 If -then 9498 , , instinct328 , , 96 production , 347 RWIrobot 197 219 316 . , , , , 330 Shaping , 154 Sheep Simulated , 155 annealing Simulations . 53. 136 168 172 333 . 27 . . . . - , , 337 342 349 379 381 428 , , Situated automata model87 89 , Situatedness26 135 , 8, , Situational , , 223 Saliency problem309 hierarchy SAMUEL338 SIVS See . Architecture , , SIVS Satellites . 249 Societal , -378 , Agent Theory376 , 412 420 Scaleissues , 28 of of , , , Society Mind 15 376 Software , 203 , Scanpaths , 277 279 Scarecrow , 171 , , agent163 170 171 SCARF , 289 , 66 , , , modularity , 129 163 166 218 Schema , objects163 reuse156 163 264 , 246 , , , anticipatory - , avoidpast 145 155 158 161 163 165 - , , , , , , 220 Sojourner 187 188 SOMASS , 231 definitions , 48, 255 Sonar Sensor . See , 43 , ultrasound , 145 , 428 , encodings - 148 Sozzy - 429 435 motor 141 165 174230 256 , , , , , 215 Spatial reasoning , 143 , , , -256 , 367 perceptual - 144215 217 254 , 407 Speedup - 368 , 434 , 276 receptor Spotlight metaphor activation Activation . See , , spreading signal434 Spreading , - , , , , , , theory41 43 163 167254 255 Squirt 133 140439 -response translnitter SRdiagrams Stimulus . See , 434 diagrams SRI(Stanford Research Institute 16 166 , 362 ), , , Schooling , 365 -Plan Sense -Act paradigm 227 231 346 , 130 , , and , passive active247 249 409 , , , , Sensing Stability432 Sensitization - 307 , 306 , 187 Stagnation , 391 Sensor Stanford , 18 19 Cart Stanford , 429 , 295 compass University OOPS249 407 Stickleback , 48- 49 Fish , , fuel 215 434 Stimulus , , fashionSee conditioned - 324 , 322 ( Perceptual ) sequencing fission268 , 299 domain91 92 , -269 , fusion268 269 283 , 299 -288 environmental , - , , 142 GPS 188248 , 407 414 , , -249 , , 241 perceptual - 242 infrared19 168 342 , , , , plane89 INS 188248 , , , 91 strength , 93 internal215 unconditioned - 324 , , 322 -response laser scanner , 251 254 Stimulus - , , 247 , 79 diagrams - 80, 118 120 models107 139 161 162 , , . STM See ,9 , photocell Memoryshortterm shaft encoders - 248 Stress , 247 , 432 , 215 , temperature Stripe401 touch313 STRIPS , 17 232 , , 14 , architecture Architecture . See ultrasonic - 19 168 188 190 193 202 - , , , 18 , , , , Subsumption 216 243 247 249 250 253 316 326 429 , , , - , , , , subsumption . See vision(See Vision computer , , ) Superpositioning Vectorsummation Survival51 whisker328 , , , 133 , , - , Symbol grounding problem27 181 182241 Seymour ,140 . Architecture SFX See , SFX T-norms115 Shaft encoders Sensor . See encoders , shaft , tissues , 16 , , , Shakey - 17 130 166 Target

490

SubjectIndex
, Targetability 434 Taxes 47- 48 , TEIRESIAS, 347 Teleautonomy145 165 377. 415- 420 -.- .- .

, Utility problem309 . UUV See Unmanned vehicles , undersea Value , 325 system VaMoRs -290 292 , 289 , VaMP289 291 , Vector action vectors144 181 , , based motor control38 41 113 114 142 , - , - , fieldhistogram , 190 fields232 , , 40 hypothesis , maps184 185 , polar 184 , 189 steering sllmmation - 99 101 113 114 142 - , , 98 , , , 149 150215 347 371 378 - , , , , Viewframes , 193 Vision -245 , , -274 , , biological35 243 , 245 257 273 , 393 -56 fly, 55 , spatial245 whatandwhere36 181 200 Wl , 243 , , , Visioncomputer , activeSee , active ( Perception ) animate Perception , active (See ) frommotion 194 , depdl functional - 202 , WI , 264 lightweight -265 -based model , 201 , 194 panoramic , 395 397 , 238 paradigms -240 with problems , 238 , actionoriented (See ) purposive Perception , 203 - , recognition , 274 275 277 -based schema , 42, 254 256 377 - , sensor - 251 , 250 situated Perception , actionoriented (See ) stereo18 35 140 189298 393 , , , , , , taskorientedSee , action ( Perception oriented ) , dleory257 - , , , - , , tracking251 252 258 274 275 286 293 297

Teleo -reactive , 118233 , 96 , , 289 Terregator , 292 Teseo , 327

Theaagent232 , ' of , Albuss, 21 22 Theory intelligence , 431 Thermoregulation - 432 , computational - 425 , 423 Thought , Thyroid431 Tito, 140 TJ, 229 -230 TomandJerry 140 , Tooth219 220 , 's Tortoise Walter , 8- 10 , Grey Toto 133 140 193 , , , . See , Tracking Vision tracking , 441 Transmigration - 442 , , Tropism8, 381 399 401 Test Thring , 423 UAVSee . Unmanned vehicles , aerial UGV See . Unmanned vehicles , ground illtrasonic sensors Sensor . See , ultrasound , illysses260 263 UM-PRS227 412 , , Universal . See , universal plans Plan of , University Alberta374 of , 53 University Brussels of , , 203 University CaliforniaBerkeley of , University CaliforniaLosAngeles ( UCLA 369 394 ), , of , , 385 University CaliforniaRiverside of , 278 University Chicago , 295 of , 328 University Edinburgh of , 43 University Genova of , 328 University Karlsruhe of , 275 University Maryland of , University Michigan410 412 of , 280 University Pennsylvania of , 272 University Rochester , 297 of California84 251 , , , University Southern 327 340 381 399 , , , of , Arlington409 410 , University Texas of , University Toronto276 of , 362 University Tsukuba , 368 of , University Virginia273 of , 271 University Wisconsin vehicles Unmanned - , aerial UAV 3, 6, 251 252 289 ( ), , ( ), ground UGV 3, 7, 170401 402 undersea ) , 3, 6, 88 89 95, 115234 - , , ( UUV , , Utility function312 318

Visual routines - 262272 , 256 , Visual search , 258 , 240

, 364 Wallaby -365 Walking , - , , gaits57 58 313 342 , 136 subsumption - 137 WASUBOT ,2

-tate , , , 112 Winner -aU35 110 , 170 Win , 3 . -I

491

Index Subject
World models(seealso Representation ) asa convenience229 , conStniction 66 , global, 241 problemswith, 107 relianceon, 210 , symbolic 21 three-dimensional238 , XAVIER, 197 198 355 , Yamabico robots 362 ,