RAI Notes On Reactive Paradigm

Reactive Paradigm Overview
Subsumption Architecture
DARSHAN INGLE
Prepared by Prof. Darshan Ingle.
Reactive Paradigm Overview
A robotic paradigm can be described by the relationship

between the three primitives of robotics: Sense, Plan, and Act.
It can also be described by how sensory data is processed and
distributed through the system, and where decisions are made.
Two Representative Reactive Architectures:
1) Subsumption
2) Potential Fields Summation
Reactive Paradigm emerged late 1980s
Reactive Paradigm still important for two reasons:

1) Robotic systems in limited task domain are still being
constructed
2) Forms the basis for the Hybrid Reactive-Deliberative Paradigm
Reactive Paradigm
Sense-act type of organization.
The robot has multiple instances of Sense-Act couplings.
The SENSE-ACT organisation does not specify how the behaviors
are coordinated and controlled.
So each behavior has its own dedicated sensing which results in
more than one behavior taking the same output from the sensor and
process it differently(via behaviors perceptual schema).
Thus, one behavior does not know what another behavior is doing
or perceiving.
Hierarchical Paradigm
The robot operates in a top-down fashion, heavy on
planning.
The robot senses the world, plans the next action, acts;
at each step the robot explicitly plans the next move.
All the sensing data tends to be gathered into one global
world model.
Hybrid Deliberate/Reactive Paradigm

The robot first plans (deliberates) how to best decompose a task into
subtasks (also called mission planning) and then what are the
suitable behaviours to accomplish each subtask.
Then the behaviours starts executing as per the Reactive Paradigm.
Reactive Paradigm grew out of the dissatisfaction of the hierarchical

paradigm and with the influx of ideas from biological intelligence.
Reference(Biological Intelligence)
To gather information requires senses, or sensors and the ability to
attach labels to the sensed perceptions and to manipulate these
labels to make predictions (algorithms, thinking).
Biological intelligence evolves out of multi-cellular bodies with
senses, nervous systems or brains (wet-ware).
Non-biological forms of intelligence require sensors, electrical
networks or computer brains (hardware).
Although various reactive systems may or may not strictly adhere to
the principles of biological intelligence, they generally mimic some
aspect of biology.
Horizontal Decomposition of Hierarchical Model

This dissatisfaction was best characterised by those systems having
Horizontal Decomposition.
Sensors
Extract
Features
SENSE
Combine
Features
into
Model
Plan
Tasks
Task
Execution
PLAN
Motor
Control
Actuators
ACT
Vertical Decomposition
1.
2.
3.
4.
5.
Instead, examination of ethological literature suggests that

intelligence is layered in a vertical decomposition:
Agent starts with primitive survival behaviours.
Evolve new layers of behaviour which either:
a) Reuse the lower, older behaviour
b) Inhibit older behaviour
c) Create parallel tracts of more advanced behaviours
Parallel tracks can be thought of as vertically stacked layers
Each layer has access to sensors and actuators independently of
any other layer.
If anything happens to the more advanced layer, lower levels
should still operate. i.e. human brain (breathing) continue
independently of higher order functions (counting, face
recognition).
8
Vertical Decomposition
Vertical decomposition of tasks into a S-A Organisation, associated with
the Reactive Paradigm
Build maps
explore
Sensors
Actuators
wander
Avoid Collisions
SENSE
ACT
SENSE
ACT
SENSE
ACT
SENSE
ACT
The major theme of this chapter is that all

reactive systems are composed of
behaviors, though the meaning of the
behavior may be slightly different in each
reactive architecture.
Behaviors can execute concurrently and/or
sequentially.
10
Attributes of Reactive Paradigm

The fundamental attribute of reactive paradigm is all actions are
accomplished through behaviours
As in ethological systems, behaviours are a direct mapping of
sensory inputs to a pattern of motor actions that are then used to
achieve a task.
From the mathematical perspective, behaviors are simply transfer
functions i.e. they transfer SENSORY INPUTS into ACTUATOR
COMMANDS.
Here, we consider BEHAVOIR as a schema which will consist of
atleast one motor schema and one perceptual schema.
Motor schema contains the algoritm for generating the pattern of
action in a physical actuator and Perceptual schema contains the
algorithm for extracting the percept and its strength.
11
Attributes of Reactive Paradigm ( Contd . . . )

The reactive Paradigm literally threw away the PLAN component of
the SENSE, PLAN , ACT triad.
The SENSE and ACT components are tightly coupled into behaviors
and all robotic activities emerge as the result of these behaviors
operating either in sequence or concurrently.
S-A organisation of the

Reactive Paradigm into
multiple, concurrent
behaviours
Behaviour
Behaviour
SENSE
ACT
Behaviour
SENSE
ACT
Behaviour
SENSE
ACT
SENSE
ACT
12
Characteristics of Reactive
Behaviours
1)
Reactive robotic systems execute rapidly (tight coupling of senses

permits real-time operation)
Behaviours can be implemented directly in hardware circuits or
low computational complexity algorithms (O(n)).
Have no memory (limiting behaviours to stimulus-response
reflexes)
Main point Behaviours controlled by what is happening in the
world, rather than the program storing & remembering what the
robot last did
Five Characteristics of reactive Paradigm are:
Situated Agent (integrated part of world)
Robots are situated agents operating in an ecological niche.
i.e. when a robot acts, it changes the world, and receives
immediate feedback about the world through sensing.
13
Characteristics of Reactive Behaviours (continued)

2) Emergent Behaviours
Behaviours serve as the basic building blocks for
robotic action, and the overall behaviour of the robot
is emergent.
Behaviours are independent computational entities and
operate concurrently, Hence there is no explicit
controller module which determines what will be done,
or which function call other functions.
3) Ego-centric / Robot centric
Only local, behaviour specific sensing is permitted.
i.e.consider OBSTACLE AVOIDANCE example.It does
not matter that an obstacle is in the world at coordinates
(x,y,z), only what matters is where it is relative to the
robot.
14
Characteristics of Reactive Behaviours

(continued)
4) Modular Behaviours
These systems inherently follow good software design
principles.
The modularity of behaviours supports the decomposition of a task
into component behaviours.
The behaviors are tested independently, and the behaviors may be
assembled from primitive behaviors.
5) Biological Motivation
Animal models of behaviour are often cited as a basis for these
systems or a particular behaviour (unlike earlier AI days where a
conscious effort to not mimic biological intelligence was made)
Thus, it is very acceptable under Reactive Paradigm to use animals
as motivation for a collection of behaviors.
15
Advantages of Programming by Behaviour

Constructing a robotic system under Reactive Paradigm often
referred to as programming by behaviour.
Programming by Behaviour has number of advantages most of
them consistent with good software engineering principles:
1. Behaviours are Modular.
2. Easy to test in isolation (Unit Testing).
3. Also supports incremental expansion of capabilities of a robot
i.e. a robot becomes more intelligent by having more behaviors.
Modules have Low Coupling
can function independently of each other with minimal
connections or interfaces promoting easy reuse.
Modules have High Cohesion
Data and operations contained by a module relate only to the
purpose of that module.
16
Reactive Paradigm Representative

Architectures
The overall action of the robot emerges from the multiple concurrent
behaviours and the architecture must provide mechanisms for:
1) Triggering behaviours
2) Conflict resolution when multiple behaviours are active at any
one time.
The two most well known Reactive Architectures are

1)
Subsumption Decomposition into layers of task achieving
behaviours.
2) Potential Fields Behaviours combined by summation of fields
17
Subsumption Architecture
Rodney Brooks Subsumption Architecture is the most influential of
the purely Reactive Paradigm systems.
These robots look like shoe-box sized insects (6 legs and antennae)
Implementations quite often have the behaviours embedded directly
in the hardware or small micro processors allowing the robots to
have all on-board computing.
Robots could now walk, avoid collisions and climb over obstacles
without the move-think-move-think pauses of Shakey.
A behaviour is a network of sensing and acting modules which
accomplish a task.
The modules are Augmented Finite State Machines (AFSM), or finite
state machines which have registers, timers & other enhancements
to permit them to be interfaced with other modules.
18
Subsumption Architecture (continued)

Behaviours are released in a stimulus-response way, without an
external program explicitly coordinating and controlling them.
Four interesting aspects of subsumption in terms of releasing and
control are:
1. Layers of Competence:
Modules are grouped as layers of competence.
The layers reflect a hierarchy of intelligence or competence.
lower layers encapsulate basic survival functions (collisions)
higher levels create more goal directed actions (mapping)
Each of the layers can be viewed as an abstract behaviour for a
particular task.
2. Subsumption of lower layers:
- Modules in a higher layer can override or subsume the output from
behaviours in the next lower layers.
- Behaviour layers operate concurrently and independently and
hence need mechanism to handle potential conflicts winner
always takes all (always
the highest layer)
19
Subsumption Architecture (continued)

3. No World Model (No Internal State)
-The use of the internal state is avoided.
-Because robot is a situated agent,most of its info comes directly
from the world
- If the robot depends on internal representation,what it believes
may begin to dangerously diverge from reality.
- Some internal state needed for releasing behaviours (i.e. scared,
hungry), but good design minimizes this.
4. Taskable
- Tasks are accomplished by activating the appropriate layer, which
then activate the lower layers.
- In practice, Subsumption style systems are not easily taskable,
that is, they cant be ordered to do another task without being
reprogrammed.
20
Level 0 AVOID Architecture: obstacle avoidance
Sensor: A SONAR module that gives the distance to the objects in polar
coordinates.
Internal Modules
COLLIDE detects if front obstacle is too close (i.e. halt signal)
FEELFORCE it treats sensor reading acts as repulsive force field.It
can be thought of as summing the vectors from each of thesonar
readings.This results in a new vector.
RUNAWAY provides direction to move
Actuators
TURN provides motor output to turn robot.It splits off the direction to
turn and passes it to steering actuators. It also passes it to the
FORWARD module which uses magnitude to determine the magnitude
of next forward motion(how far/how fast)
FORWARD switches forward motion on or off
21
Sonar module reads the sonar ranges.

Polar plot range readings in polar coordinates (r,)
surrounding the robot.
7
1
6
2
3
robo-centric view of range readings
Unrolled into a plot
22
Level 0 Architecture Diagram
23
Level 1 Architecture: WANDER

Wander - Cant pass directly onto Turn
since will sacrifice obstacle avoidance
How does Turn know which

module to take heading from?
24
Level 1 (Contd)
Consider a robot which actually wandered around instead of sitting
motionless, but was still able to avoid obstacles.
This is achieved using WANDER module
-which computes the random headings(vector) every n seconds
-pass it to TURN and FORWARD module
But, it cannot be passed to TURN module directly since TURN
accepts only one input.
One solution is to add another module in level 1, AVOID, which
combines FEELFORCE vector with WANDER vector.
This also offers an opportunity to create a more sophisticated
response to obstacles.
AVOID combines the direction of force of avoidance with desired
heading which results in actual heading being mostly in right
direction.This is now fed to TURN module.
Note:AVOID module was able to EAVESDROP on components of
next lower layer.
25
Level 1 (Contd)
The issue now appers to be when to accept the heading vector from
which layer.
Subsumption makes it simple: the o/p from the higher layer
subsumes the o/p from the lower layer
This is done in one of the two ways:
1. Inhibition
O/P of subsuming module is connected to O/P of another
module.
If O/P of subsuming module is ON/has any value-> O/P of
subsumed moduleis blocked/turned OFF.
2. Suppression
O/P of subsuming module is connected to I/P of another
module.
If O/P of subsuming module is ON,it replaces normal I/P to
subsumed module.
26
Level 1 (Contd)
In this case, AVOID module supresses o/p from RUNAWAY.

RUNAWAY is still executing but its o/p doesnt go anywhere.
Instead, o/p from AVOID goes to TURN.
The use of layers and subsumption allows new layers to be built on
top of less competent layers, without modifying the lower layers.
This facilitates MODULARITY and simplifies TESTING.
It also adds some ROBUSTNESS i.e. if something disables level1
behjavoir, level0 behavior is still intact. Thus the robot would atleast
be able to preserve its self-defense of fleeing from approaching
obstacles.
27
Level 2 Architecture: follow corridor

Integrate - estimates how far robot has travelled off course
Supplies dangerous internal state
28
Level 2 (Contd)
Third layer is used to permit a behavior move down the corridor.
LOOK module-> examines the sonar polar plot & identifies the
corridor.
LOOK module isnt included in the lower layer since identifying the
corridor is more computationally expensive than just extracting
range data.Thus it would have taken longer time to execute at lower
layer.
STAYIN MIDDLE module-> LOOK passes the vector representing
the direction of middle to STAYIN MIDDLE module.
Now this STAYIN MIDDLE module subsumes the WANDER module
and delivers the o/p to AVOID module.
Thus the robot can now swerve around obstacles.
How does the robot get back on course if the LOOK module has not
computed a new direction?The solution is INTEGRATE module
29
Level 2 (Contd)
INTEGRATE module->observes the robots actual motions from
shaft encoders in actuators.This gives an estimate of how far the
robot has travelled since the last update by LOOK.
Thus INTEGRATE module supplies a dangerous internal state i.e. it
substitites for the feedback from the real world.
If for some reason , the LOOK module never updates,then the robot
would operate without any sensor data forever/atleast until it is
crashed.
So subsumption style systems include time constants on supression
& inhibition.
If the supression from STAYINMIDDLE ran for longer than n
seconds without a new update, the supression would cease.
The robot would then begin to wander, and hopefully whatever
problem(like the corridor being totally blocked) that had led to the
loss of signal would fix itself.
30
Potential Fields Methodologies(Visualizing

potential fields)
Potential Field styles of behavior always use vectors to represent
behaviors & vector summation to combine vectors from different
behaviors to produce an emergent behavior.
Potential field-> is an array or field of vectors.
Vector->is a mathematical construct which consists of a mafnitude &
direction.
The array represents a region of space. It is represented in 2dimensions, representing a birds eye view of the world,just like a
map.Map can be divided into squares creating a (x,y) grid.Each
element of the array represents a square of the space.
Perceivable objects in the world exert a force fieldanalgous to
magnetic or gravitational fied) on surrounding space
31
Visualizing potential fields( Contd)
32
5 Types of potential fields
33
5 Types of Potential fields(Contd )

1.
2.
3.
Uniform:
Here , the robot would feel the same force no matter where it is.
From any position or orientation, the robot would align itself to the
direction the arrow points and will move in that direction at a
velocity proportional to the length of the arrow.
Perpendicular:
Here, the robot is oriented perpendicular to the object/wall/border.
The field is directed away from gray wall, but perpendicular field
can be pointed towards the object as well.
Attractive:
Here, whereever the robot is, it will feel a force relative to the
object.
It is usually used when the agent is literaly attracted to
light/food/goal.
34
5 Types of potential fields(Contd)

4.
5.
Repulsive:
Opposite of attractive field
These are normally associated with obstacles/things that the agent
should avoid.
Closer the robot is to the object,stronger the repulsive force,180
degree away from it.
Tangential:
This field is tangent around the object.
They can spin either clockwise/anti clockwise.
They are useful for directing the robot to go around the
obstacle/having a robot investigate something.
35
Magnitude Profiles
Notice slide
33(fig c & fig
e),length of
the arrow
get smaller
closer to the
object.
The way of
changing of
this
magnitude
of the
vectors is
called
magnitude
profile.
36
Magnitude Profiles(Contd)
1.
2.
3.
Constant magnitude:
When a robot encounters a robot a obstacle, it either tries to run
away from it or stops there.
This whole process is happening at a constant speed,which leads
to jerky motion.
Linear drop off:
When a robot encounters a robot encounters a obstacle,it will turn
and move quickly towards it ,then slow up to keep from over
shooting & hitting the object.
Exponential drop off:
Here, the drop off is proportional to the square of the distance.
Thus, for every unit of the distance away from the object, the force
on the robot drops in half.
37
Combination of fields & behaviors
The first attribute of true potential field methodology is that it requires all
behaviors to be implemented as potential fields.
The second attribute is that it combines behaviors by vector summation.
Now, how are this behaviors implemented & combined?
38
Combination of fields & behaviors(Contd)

The motor schema of MOVE2GOAL is represented with attractive
potential field. It uses shaft encoders on the robot to tell if it had
reached the goal position.
The RUNAWAY behavior is the repulsive behavior. It uses a range
sensor to detect if something is in front of it.
Combined field behavior->
1. Robot starts at lower right corner.
2. At time t0,the robot senses the goal. It can only perceives the goals
& cannot perceive the obstacle.So, it feels only attraction vector at
the start i.e. RUNAWAY vector has magnitude 0.0
39

3.
4.
5.
6.
7.
It moves along a straight line for the goal.

At time t2, it updates it sensors and now perceives both the goal
and the obstacle..
Thus both the behaviors contribute a vector.these are summed
and the robot now moves offcorse.
At time t3, the robot has almost moved beyond the obstacle. Thus
goal exerts a stronger force now.
At time t4, it resumes cousrse and reaches the goal.
40
41
42

Some disadvantages of it are that it suffers from the following problems:
1. Update Rate:
o
The distance (length of the arrows)between updates is different. This
is due to the changes in the magnitude of the outer vector, which
controls the robot velocity.
o If it has shorter range vector->it moves slowlyi.e. covers less distance
in the same amount of time.
o If it has longer range vector->it moves faster(i.e. it overshoots between
t3 & t4 where the robot actually goes farther without turning & has to
turn back to the goal).
o As a result , the path is jagged with sharp lines.
o The resulting path would have been smoother if the robot has a faster
update interval.
43

2. Local minima:
o Along the line behind the obstacle, the vectors have only head
i.e. direction of the arrow & no body i.e. length of the arrow(refer
slide no. 41)
o This means that the potential fields have summed upto zero i.e.
magnitude has become zero
o So if our robot reaches at this spot, it will stop and not move
again.
o This is called as the local minima problem.
o This is because potential fields have minima or valley, that traps
the robot.
44

3. Holonomicity is the relationship between the controllable degrees
of freedom of the robot and the total degrees of freedom of the
robot.
o If the number of controllable degrees of freedom are greater than
or equal to the total degrees of freedom a robot is said to be
holonomic.
o By using a holonomic robot many movements are much easier to
make and returning to a past pose is much easier.
o Potential Field Planning places values over the map with the goal
having the lowest value raising the value depending on the
distance from the goal.
o Obstacles are defined to have an incredibly high value. The robot
then simply moves to the lowest potential value adjacent to it,
which should lead it to the goal. However this technique often gets
trapped in local minima.
45
Example using one behavior per sensor

Robot is able to get
out of the cave-like
trap called as the
BOX CANYON.
It does so without
building a model of
the wall.
Each instance
contributes a
vector.,some of which
have X or Y
component that
cancel out.
46
Potential fields compared with Subsumption

In level 0 in subsumption , if there are no obstacles within range,
the robot feels no repulsive forces and is motionless.
If an obstacle comes within range & is detected by more than one
sonar,each of the sonar readings create a vector , pointing the robot
in opposite direction.
The COLLIDE module in the subsumption does not map over ton a
behavior in potential field methodology.
COLLIDE is to stop the robot if it touches an obstacle i.e. if the
RUNAWAY behavior has failed.
But it does not produce any potential field, unless a uniform field of
magnitude zero is permissible.
So, COLLISION vector would have no impact.
Infact, collisions are often treated as panic situations which trigger
an emergency response.
47
Potential fields compared with Subsumption(Contd)
48

In level1 , WANDER behavior
generates a new direction to
move every n seconds.
RUNAWAY behavior
generates a behavior which
will cause a robot to move 180
degree away from the
obstacle.
By combining the o/p from
WANDER with the o/p from
RUNAWAY , the need for
AVOID behavior is eliminated.
The robot now moves in the
desired direction i.e. resultant
direction.
49
50

In level 2, the FOLLOWCORRIDOR behavior causes the
robot to stay in the middle of the
corridor.
This is accomplished using two
potential fields:
1. A uniform field perpendicular to the left
boundary & pointing to the middle.
2. A uniform field perpendicular to the
right boundary & pointing to the middle.
However, these two uniform fields

are not sufficient for the robot to
move forward.
So a third uniform field is added
which is parallel to the corridor.
All three fields combined yield a
smooth field which sharply pushes
the robot back to middle of the
corridor.
51
1.
2.
3.
4.
5.
Docking behavior:
Docking is when a robot moves to a specific
location& orientation relative to docking
station.
This is useful for robots performing material
handling in industry.
So in order to accept a piece of material to
carry,the robot has to be close enough to the
correct side of the end of conveyer and
facing the right way(i.e. in right position &
orientation)
So we go for selective attraction field. It
makes the robot feel the attractive forces
when it is within certain angular range of
docking station.
However , it does not cover the case of when
the robot approaches from behind or to the
side.
52

6.
7.
So we go for
tangential field.It
makes the robot
orbit the dock until it
gets the selective
attraction area.
This combination of
the 2 motor
schemas produces
very smooth field
which keeps the
robot in correct
position &
orientation.
53
54
Advantages of potential field styles

1.
2.
3.
4.
5.
It is a continous representation which is easy to visualize over a

large region of space.
Thus its also easy for the designer to visualize the robots overall
behavior.
Its also easy to combine fields & languages like C++ support
making behaviorial libraries.
The potential field can be parameterised i.e. the range of
influence can be limited and any continous function can express
the change in magnitude over distance(linear,exponential,etc.)
2D fields can be usually extended into 3D fields, & so behaviors
developed for 2D will work for 3D.
55
Disdvantage of potential field styles
1.
2.
Multiple fields can sum upto a vector with zero magnitude->local

minima problem.This causes the robot to become stationary.
Solutions to local minima problem:
One solution is to always have a motor schema producing vectors
with a small magnitude from random noise.
This noise in the motor schema would serve to bump the robot off
the local minima.
A more recent solution is to express the fields as harmonic
functions. These guarantee to not to have a local minima of 0.
The only disadvantage of this technique is that it is
computationally expensive & has to be implemented on a VLSI
chip in order to run in real-time for large areas.
56
Disdvantage of potential field styles( Contd)

3. Another solution is using Navigation Templates(NaTs)
--The local minima problem occurs due to the avoid behavior.
--However, in NaTs, avoid behavior receives as input the vector
summed from the other behaviors.
--This vector represents the direction the robot would go if there
were no obstacles nearby. We refer this vector as the strategic
vector.
--If the robot has strategic vector, this vector gives a clue as to
whether an obstacle should be passed on the right or the left.
--eg:A robot crossing the bridge,it will be passing to the left of the
obstacles on the right in order to stay in middle.
--The o/p of this avoid behavior is called as the tactical vector as it
takes the takes the robot in the right direction so as to achieve the
strategic goal.
57
THANK YOU.
Prepared by Prof.
58 Darshan Ingle.
Designing a Reactive
Implementation
DARSHAN INGLE
59
Overview
By this time, the sheer simplicity & elegance of reactive robots incite
people to start designing and implementing their own robots.
Several kits(Lego Mindstorms, Rug Warrior) permit the rapid
coupling of sensors & actuators, enabling users to build reactive
behaviors.
However new issues are how to program more intelligence into
software and how to exploit better sensors than those which come
with kits.
Unfortunately, good intentions in robot programming are often
frustrated by two deficits:
1.
2.
Designing behaviors tend to be an art, not a science.Novice roboticists

are often uncertain as to how to even start the design process.
Once the designer has few well-designed and tested behaviors, how
are they integrated into a system?
60
Overview (Contd)
Emergent behavior-> When a small set of behaviors are combined
internally to produce an overall behavior.
Series of behavior-> Here, behaviors are operating in a recognizable
sequence.
This chapter attempts to aid the novice designer in constructing a
reactive robot system by addressing each of these two deficits:
1. An object oriented approach to designing behaviors is
introduced.
2. Two techniques for managing behaviors of robot are
introduced:
Finite state automata

Scripts.
61
Behaviors as Objects in OOP

Object consists of data and methods, also called as attributes &
operations.
Schemas( refer slide no. 63):
1. Schemas contain specific knowledge & local data structure & other
schemas.
2. A schema as a programming object will be a class.
3. The class will have an optional method called as the coordinated
control program.
4. Coordinated control program is a function that coordinates any
methods or schemas in derived class.
Behaviors ( refer slide no. 64):
1. Behaviors are composed of atleast one perceptual schema and one
motor schema.
2. These schemas act as method for the behavior class.
62
Behaviors as Objects in OOP->Classes-Schema

( Contd )
63
Behaviors as Objects in OOP->Classes-Behavior

( Contd )
64
Behaviors as Objects in OOP ( Contd)

3. Perceptual schema has atleast one method, which takes sensory
i/p & transforms it into data structure called a percept.
4. Motor schema has atleast one method, which transforms the
percept into a vector or other form of representing an action.
The OOP organization allows a behavior to be composed of
multiple perceptual schema & motor schema and even behaviors.
Primitive behavior:
It is composed of only one perceptual schema and one motor
schema.
They are monolithic i.e. they do only one thing.
They are simple mapping from stimulus to response.
Hence , they are often programmed using single methods or
objects.
65
Behaviors as Objects in OOP ( Contd )

Abstract behaviors:
These are the behaviors which are assembled from other behaviors
or have multiple perceptual schema or motor schema.
The term of abstract behavior should not be confused with an
abstract class in OOP.
66
Example: Primitive move-to-goal behavior

Pick up the Trash event(refer slide no. 68):
The robot was placed in an empty arena about the size of an office.
The arena would have Coca-Cola cans & white stryofoam cups
paced at random locations.
In two of the four corners, there would be a blue recycling , in the
other two, a different colored trash bin.
The robot who picked up the most trash and placed them in correct
bins, in allotted time was the winner.
In most years, strategy was to find & recycle the Coco-Cola cans
first, because it was easier for the robots vision processing
algorithm to perceive red and blue.
One of the most basic behaviors needed for picking up the red soda
can and moving to the blue bin is move_to_goal.
67
Example: Primitive move-to-goal behavior( Contd )
68

When the robot sees the red can, it must move to it.
When it has a can , it must find & then move to blue bin.
It is better to write a general move_to_goal behavior, where only
what is goal- a red or a blue region-varies.
The goal for the current instance can be passed in as instantiation
through the object constructor.
So we write a single generic behavior move_to_goal(color) instead
of writing move_to_red and a move_to_blue behavior.
This behavior of move_to_goal would consist of a perceptual
schema, which is called as the extract-goal & the motor schema,
which uses an attractive field called pfields.attraction.extract-goal
which uses the affordance of the color to extract where the goal is in
the image. It then computes the angle to the center of the colored
region and the size of the region.
The affordance of the coke can is the color , while the information
extracted from the perception is the angle & the size.
69
The above table implies some very important points about

programming with behaviors:
1. Behavior is the glue between the perceptual & motor schema i.e. both
the schemas are independent & they do not communicate.
2. Behaviors can( and should ) use libraries of schema.
3. Behaviors can be reused if written properly. In this example,
move_to_goal behavior was written to accept a structure( or object)
defining a color & then moving to a region of that color i.e. the
behavior can be used with both red Coke cans & blue trash cans.
70
Example: An abstract follow corridor behavior

The move_to_goal example used the potential field methodology
using a single motor schema with a single perceptual schema.
The follow_corridor potential field consists of 2 primitive fields:
Two instances of perpendicular to the walls
One uniform parallel to the walls.
The follow_corridor field could be implemented in schemas in

atleast two different ways:
1. Each primitive field would be a separate motor schema(refer slide
73 fig 5.2 a)
The follow corridor motor schema consists of 3 primitives & the
coordinated control program..
The coordinated control program would be the function that knows
that one field is going towards the center of the corridor, which way
is forward, etc.
These are summed up by the coordinated control program in the
behavioral schema to produce a single o/p vector.
71

The perceptual schema for the follow_corridor would examine the
sonar polar plot and extract the relative location of the corridor walls
& then return the distance to the left wall and the right wall.
2. Another way to have achieved the same overall bahavior is to have
follow_wall composed of 2 instances of follow_wall behavior:
follow_wall(left)
follow_wall(right)
Each instance of follow_wall would receive the sonar polar plot and
extract the relevant wall as shown in the associated class
diagram on slide no. 74 fig 5.2 b
In both the behaviors , motor schema ran continously and the
vectors were summed internally in order to produce a single o/p
vector.
Since there were multiple motor schemas, the coordinated control
program for follow_corridor behavior is not null as it was for
move_to_goal.
Thus the vector summation & the concurrency form the conceptual
coordinated control program
in this case.
72
73
74
Where do releasers go in OOP?

The previous examples showed how behaviors can be implemented
using OOP constructs, such as classes.
Another important of the behavior is how it is activated.
Perception serves two purposes:
To release a behavior and
To guide it.
But what object or construct contains the releaser and how is it

attached to the behavior?
The answer to the first part of the question is that the releaser is
itself a perceptual schema.It can execute independently of whatever
else is happening with the robot, it is perceptual schema not bound
to the motor schema.
Eg: The robot is looking for the red coke cans with extract_color
perceptual schema.One way to implement this is when the schema
sees red,It can signal the main program that there is red. The main
program can determine now that the releaser for the move_to_goal
behavior has been satisfied.In this approach, the main program is
responsible for calling the right objects at the right time.The releaser
Preparedby
by the
Prof. designer.
Darshan Ingle.
75
is attached to the behavior
Where do releasers go in OOP? ( Contd )

So we have two move_to_goal objects , one instantiated with goal of
red and the other with goal of blue.
Thus, the move_to_goal behavior can use any perceptual schema
that can produce a goal angle and a goal strength.
If we want to program the robot to move to bright light
(photomorphism), only the perceptual schema would need to be
changed..
This is an example of Software Reusability.
76
Steps in designing a Reactive Behavioral System
77
Steps in designing a Reactive Behavioral System( Contd...)

This methodology assumes that a designer is given a task for a
robot to do, & a robot platform.
The goal is to design a robot as a situated agent.
Therefore, the first three steps serve to remind the designer to
specify the ecological niche of the robot.
Fourth step begins a iterative process of identifying and refining a
set of behaviors for the class.
It asks questions:
what does the robot do?
How it acts & reacts to the range of variability in the ecological niche.
Eg: In Pick Up Trash event, most of the teams used partitioning

along the lines of : Random search until see red, move to red, pick
up can,random search until see blue,move to blue, drop can.
Steps 5-7 are less abstract.
Here we specify the motor & the perceptual schemas.
78
Steps in designing a Reactive Behavioral System( Contd...)

This is where the designer has to write a algorithm for finding red
blobs in a camera image for the random search until find red and
move to red behavoirs.
The designer usually does this as follows:
Programs each schema independently
Integrates them into a behavior and
Tests the behaviors in isolation before integrating all behaviors.
This style of testing is consistent with good s/w engineering

principles, and also emphasizes the practical advantages of the
Reactive Paradigm.
79
Assemblages Of Behaviors
There are three common wys of representing how a sequence of
behaviors: finite state automata, scripts & skills.
Finite state automata and scripts are logically equivalent, but they
differ slightly in their implementation way.
Skills collect behavior-like primitives called Reaction-Action
Packages(RAPs) into a sketchy plan which can be filled in as the
robot executes.
80
Finite State Automata

FSA are used for specifying what a given program should be doing
at a given time or circumstance.
It is written as a table or drawn as a state diagram, thereby giving
the designer a visual representation.
States: The designer has to be able to specify a finite number of
discrete states, that the robot should be in ( represented by K ).
And each state qK is a listing of behaviors that should be active at
the same time.
States are represented in the figure under the heading q, and by
circles in the graph.
Start State: It is the state at which the robot starts.( represented as s
or q0 and drawn with double circle).
Inputs/ Alphabet: These are the behavioral releasers and appear
under the column heading .FSM table considers what happens to
each state q for all possible inputs.
81
Finite State Automata ( Contd )

Transition Function: called specifies what state the robot is in
after it encounters an input stimulus ,.
A set of stimulus or affordances that can be recognized by the
robot is represented by ( a capital ).
These stimuli are represented by arrows.
Each arrow represents a releaser for a behavior. The new behavior
triggered by the releaser depends on the state the robot is in.
Final State: It represents the state when the robot has completed its
task.
Each state that the robot can reach that terminates the task is a
member of the set of final state, F.
82

Thus, the FSA table is an extension of the behavioral table. This
resulting table is known as finite state machine abbreviated as M.
The notation is:
M={K, , , s,F}
The table below summarizes the relationships of FSA to behaviors:
83

Unmanned Ground Robotics Competition:
The objective of this competition was to have a small unmanned
vehicle ( no larger than a golf cart) to autonomously navigate around
an outdoor course of white lines painted on grass.
In case of this UGR , there were only two states:
Following the line &
Moving forward
Follow-line behavior was the start state.

There are just two releasers for the UGR example, so the table does
not have many rows.
Also , there is no final state ,since the robot is never done i.e. it
needs to be turned off manually or it runs out of power.
Thus, both the states are final states.
If our robot could recognize the finish line, then it could have a finish
state. It could be just stopped, or it could be another behavior, such
as a victory wag of a camera.
However, this adds more rows to the FSA.
84
85
86
Abstract Behaviors
FSA are useful tool for expressing the coordinated control program
of an abstract behavior.They fall short as a programming tool for
abstract behaviors in a number of ways.
First in many cases,the assemblages of behavior represents a
prototypical sequence of events that should be slightly adapted to
different situations i.e. a TEMPLATE or a ABSTRACT BEHAVIOR.
Eg: In the Pick Up The Trash event, recycling the coke cans was not
the only the part of task,the robots were also supposed to pick up
white Styrofoam cups and deposit them into yellow trash
cans.These behaviors represented by the FSA could be collected
into an abstract behavior:
pick-up-the-trash( trash-color, trash can-color, size-trash can)
Second, templates need to handle different initialization conditions.
Eg: In the start state itself, the robot is facing a bale/obstacle.
87
Abstract Behaviors ( Contd )

Another common initialization behavior is imprinting, where the
robot is presented with an object and then records the perceived
color(or other color) of the object for use with the nominal behavior.
Eg: In the Pick Up Trash competition, several teams literally showed
the robot the Coke can and let it determine the best values of red
for current lighting conditions.
Third, sometimes robot fail in their task,these events are called a
exceptions.
Eg: An exception might be when a robot does not pick up a soda
can in over 10 minutes.In this case, another behavior can be
substituted for the exception behavior i.e. we stop the random
wander & start a Raster scan instead.
88
Scripts
Abstract behaviors often use scripts, or a related construct called
skills, to create generic templates of assemblages of behavior.
Scripts encourage the designer to think of the robot and the task
literally in terms of screen play.
Scripts were originally used in Natural Language Processing (NLPs)
to help the audience(a computer) understand actors (people talking
to the computers or writing summaries of what they did).
In case of robots, actors are the robots reading the scripts.
The script has more room for improvisation though, if the robot
encounters an unexpected condition, the robot begins with the subscript . This main sequence of events is called as the casual chain.
In programming robots, we often abbreviate the routine portions of
control and concentrate on representing and debugging the
important sequence of events.
FSA force the designer to consider and enumerate every possible
transition, while Scripts simplify the specification.
89
THANK YOU.
Prepared by Prof.
90 Darshan Ingle.

RAI Notes On Reactive Paradigm

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

RAI Notes On Reactive Paradigm

Transféré par

Droits d'auteur :

Formats disponibles

Reactive Paradigm Overview

Reactive Paradigm Overview

A robotic paradigm can be described by the relationship

Reactive Paradigm still important for two reasons:

Prepared by Prof. Darshan Ingle.

Prepared by Prof. Darshan Ingle.

Hybrid Deliberate/Reactive Paradigm

Reactive Paradigm grew out of the dissatisfaction of the hierarchical

Prepared by Prof. Darshan Ingle.

Prepared by Prof. Darshan Ingle.

Horizontal Decomposition of Hierarchical Model

Prepared by Prof. Darshan Ingle.

Instead, examination of ethological literature suggests that

Prepared by Prof. Darshan Ingle.

The major theme of this chapter is that all

Prepared by Prof. Darshan Ingle.

Attributes of Reactive Paradigm

Attributes of Reactive Paradigm ( Contd . . . )

S-A organisation of the

Prepared by Prof. Darshan Ingle.

Reactive robotic systems execute rapidly (tight coupling of senses

Characteristics of Reactive Behaviours (continued)

Characteristics of Reactive Behaviours

Prepared by Prof. Darshan Ingle.

Advantages of Programming by Behaviour

Prepared by Prof. Darshan Ingle.

Reactive Paradigm Representative

The two most well known Reactive Architectures are

Prepared by Prof. Darshan Ingle.

Subsumption Architecture (continued)

Subsumption Architecture (continued)

Level 0 AVOID Architecture: obstacle avoidance

Prepared by Prof. Darshan Ingle.

Sonar module reads the sonar ranges.

robo-centric view of range readings

Unrolled into a plot

Prepared by Prof. Darshan Ingle.

Level 0 Architecture Diagram

Prepared by Prof. Darshan Ingle.

Level 1 Architecture: WANDER

How does Turn know which

In this case, AVOID module supresses o/p from RUNAWAY.

Prepared by Prof. Darshan Ingle.

Level 2 Architecture: follow corridor

Prepared by Prof. Darshan Ingle.

Prepared by Prof. Darshan Ingle.

Prepared by Prof. Darshan Ingle.

Potential Fields Methodologies(Visualizing

Prepared by Prof. Darshan Ingle.

Visualizing potential fields( Contd)

Prepared by Prof. Darshan Ingle.

5 Types of potential fields

Prepared by Prof. Darshan Ingle.

5 Types of Potential fields(Contd )

Prepared by Prof. Darshan Ingle.

5 Types of potential fields(Contd)

Prepared by Prof. Darshan Ingle.

Prepared by Prof. Darshan Ingle.

Combination of fields & behaviors

Prepared by Prof. Darshan Ingle.

Combination of fields & behaviors(Contd)

Prepared by Prof. Darshan Ingle.

Combination of fields & behaviors(Contd)