Académique Documents
Professionnel Documents
Culture Documents
atural selection tends to eliminate animals what is sometimes referred to as an optimal foraging pol-
with poor foraging strategies (methods icy (such terminology is especially justified in cases where
for locating, handling, and ingesting food) the models and policies have been ecologically validated).
and favor the propagation of genes of those Optimization models are also valid for social foraging
animals that have successful foraging where groups of animals cooperatively forage.
strategies since they are more likely to en- Here, we explain the biology and physics underlying the
joy reproductive success (they obtain enough food to en- chemotactic (foraging) behavior of E. coli bacteria (yes, the
able them to reproduce). After many generations, poor ones that are living in your intestines). We explain a variety
foraging strategies are either eliminated or shaped into of bacterial swarming and social foraging behaviors and dis-
good ones (redesigned). Logically, such evolutionary princi- cuss the control system on the E. coli that dictates how for-
ples have led scientists in the field of foraging theory to aging should proceed. Next, a computer program that
hypothesize that it is appropriate to model the activity of emulates the distributed optimization process represented
foraging as an optimization process: A foraging animal takes by the activity of social bacterial foraging is presented. To il-
actions to maximize the energy obtained per unit time spent lustrate its operation, we apply it to a simple multi-
foraging, in the face of constraints presented by its own ple-extremum function minimization problem and briefly
physiology (e.g., sensing and cognitive capabilities) and en- discuss its relationship to some existing optimization algo-
vironment (e.g., density of prey, risks from predators, physi- rithms. The article closes with a brief discussion on the po-
cal characteristics of the search area). Evolution has tential uses of biomimicry of social foraging to develop
balanced these constraints and essentially engineered adaptive controllers and cooperative control strategies for
autonomous vehicles. For this, we provide some basic ideas
The author (passino@ee.eng.ohio-state.edu) is with the Department and invite the reader to explore the concepts further. Hence,
of Electrical Engineering, The Ohio State University, 2015 Neil Ave- this article should be thought of as an introduction to some
nue, Columbus, OH 43210-1272, U.S.A. interesting biological phenomena that suggest new types of
0272-1708/02/$17.002002IEEE
52 IEEE Control Systems Magazine June 2002
optimization methods; the relevance to control systems lytical methods can provide an optimal foraging policy
needs to be further investigated, and thorough compari- that specifies how foraging decisions are made (tradition-
sons to the vast literature on global optimization remain to ally, dynamic programming formulations have been used
be done. [1]). There are quantifications of what foraging decisions
must be made, measures of currency (the opposite of cost),
Foraging and constraints on the parameters of the optimization. For
instance, researchers have studied how to maximize
Elements of Foraging Theory
long-term average rate of energy intake where only certain
Foraging theory is based on the assumption that animals decisions and constraints are allowed. Constraints due to
search for and obtain nutrients in a way that maximizes incomplete information (e.g., due to limited sensing capabil-
their energy intake E per unit time T spent foraging. Hence,
ities) and risks (e.g., due to predators) have been consid-
they try to maximize a function like
ered. Essentially, these optimization approaches seek to
construct an optimal controller (policy) for making foraging
E
decisions. Some biologists have questioned the validity of
T
such an approach, arguing that no animal can make optimal
decisions (see the references in [1]). However, the optimal
(or they maximize their long-term average rate of energy in-
foraging formulation is only meant to be a model that ex-
take). Maximization of such a function provides nutrient
plains what optimal behavior would be like. In fact, some re-
sources to survive and additional time for other important
searchers have shown that foraging decision heuristics are
activities (e.g., fighting, fleeing, mating, reproducing, sleep-
used very effectively by animals to approximate optimal
ing, or shelter building). Shelter-building and mate-finding
policies, given the physiological (and other) constraints
activities sometimes bear similarities to foraging. Clearly,
that are imposed on the animal [1].
foraging is very different for different species. Herbivores
generally find food easily but must eat a lot of it. Carnivores
generally find it difficult to locate food but do not have to eat Search Strategies for Foraging
as much since their food is of high energy value. The envi- In one approach to the study of foraging search strategies
ronment establishes the pattern of nutrients that are avail- [2], predation is broken into components that are similar for
able (e.g., via what other organisms are nutrients available, many animals. First, predators must search for and locate
geological constraints such as rivers and mountains, and prey. Next, they pursue and attack the prey. Finally, they
weather patterns), and it places constraints on obtaining handle and ingest the prey. The importance of various
that food (e.g., small portions of food may be separated by components of foraging behavior depends on the relation-
large distances). During foraging there can be risks due to ship between the predator and the prey. If the prey is larger
predators, the prey may be mobile so it must be chased, and than the predator, then the pursuit, attack, and handling can
the physiological characteristics of the forager constrain its be most important. The prey may be easy to find, but the
capabilities and ultimate success. preys size gives it an advantage. If the prey is smaller than
For many animals, nutrients are distributed in patches the predator, then generally the search component of forag-
(e.g., a lake, a meadow, a bush with berries, a group of trees ing is most important. Small size can be an advantage for the
with fruit). Foraging involves finding such patches, deciding prey. Since prey are often smaller than predators for many
whether to enter a patch and search for food, and whether animals, they must be consumed often and in large num-
to continue searching for food in the current patch or to go bers; this makes the search time limit other components of
find another patch that hopefully has a higher quality and the predation cycle. In this article, we consider cases where
quantity of nutrients than the current patch. Patches are the searching behavior is the dominant factor in foraging.
generally encountered sequentially, and sometimes great ef- This is the case for many birds, fish, lizards, and insects.
fort and risk are needed to travel from one patch to another. Some animals are cruise or ambush searchers. For
Generally, if an animal encounters a nutrient-poor patch, the cruise approach to searching, the forager moves contin-
but based on past experience it expects that there should be uously through the environment, constantly searching for
a better patch elsewhere, then it will consider risks and ef- prey at the boundary of the volume being searched (tuna
forts to find another patch, and if it finds them acceptable, it fish and hawks are cruise searchers). In ambush search, the
will seek another patch. Also, if an animal has been in a forager (e.g., a rattlesnake) remains stationary and waits for
patch for some time, it can begin to deplete its resources, so prey to cross into its strike range. The search strategies of
there should be an optimal time to leave the patch and ven- many species are actually between the cruise and ambush
ture out to try to find a richer one. It does not want to waste extremes. In particular, in saltatory search strategies, an
resources that are readily available, but it also does not animal will intermittently cruise and sit and wait, possibly
want to waste time in the face of diminishing energy returns. changing direction at various times when it stops and possi-
Optimal foraging theory formulates the foraging problem bly while it moves. To envision this strategy, consider Fig. 1,
as an optimization problem and via computational or ana- where distance traveled in searching is plotted versus time.
hrepellant = dattract i= 1 m= 1
S
p
+ hrepellant exp wrepellant (m im )2
be the height of the repellant effect (magnitude of its effect) i= 1 m= 1
and
denote the combined cell-to-cell attraction and repelling ef-
wrepellant = 10
fects, where = [1 ,K ,p]T is a point on the optimization do-
main and im is the mth component of the ith bacterium
be a measure of the width of the repellant. The values for
position i (for convenience we omit some of the indices).
these parameters are simply chosen to illustrate general
An example for the case of S = 2 and the above parameter
values is shown in Fig. 4. Here, note that the two sharp peaks
Nutrient Concentration (Valleys = Food, Peaks = Noxious) represent the cell locations, and as you move radially away
from the cell, the function decreases and then increases (to
model the fact that cells far away will tend not to be at-
5
tracted, whereas cells close by will tend to try to climb down
4
3 the cell-to-cell nutrient gradient toward each other and
2 hence try to swarm). Note that as each cell moves, so does
z=J
1 its J cci (,i ( j , k ,l )) function, and this represents that it will re-
0
1 lease chemicals as it moves. Due to the movements of all the
2 cells, the J cc (, P( j , k ,l )) function is time varying in that if
3 many cells come close together there will be a high amount
4
30 of attractant and hence an increasing likelihood that other
25
20 30 cells will move toward the group. This produces the swarm-
15 25
10 20 ing effect. When we want to study swarming, the ith bacte-
y= 15
2 5 10 rium, i = 1,2,K , S , will hill-climb on
0 0
5 x = 1
S
20 Sr =
25 2 (1)
20 15
be the number of population members who have had suffi-
cient nutrients so that they will reproduce (split in two) with
Figure 4. Cell-to-cell chemical attractant model, S = 2. no mutations. For reproduction, the population is sorted in
For initialization, you must choose p, S, N c , N s , N re , N ed , ped , ii) While m < N s (if have not climbed down too long)
and the C ( i ), i = 12 , ,K , S. If you use swarming, you will also Let m = m + 1.
have to pick the parameters of the cell-to-cell attractant If J( i , j + 1,k , l ) < J last ( if d oing b e tte r) , let
functions; here we will use the parameters given above. J last = J ( i , j + 1,k , l ) and let
Also, initial values for the i , i = 12 , ,K , S, must be chosen.
Choosing these to be in areas where an optimum value is ( i)
i ( j + 1,k , l ) = i ( j + 1,k , l ) + C ( i )
likely to exist is a good choice. Alternatively, you may want ( i ) ( i )
T
b) Compute J ( i , j ,k , l ). Let J ( i , j ,k , l ) = J ( i , j ,k , l ) + i
J health = J ( i , j ,k , l )
J cc ( i ( j ,k , l ), P ( j ,k , l )) (i.e., add on the cell-to-cell at- j= 1
order of ascending accumulated cost (higher accumulated nutrients and allows us to keep a constant population size,
cost represents that a bacterium did not get as many nutri- which is convenient in coding the algorithm.
ents during its lifetime of foraging and hence is not as Let N ed be the number of elimination-dispersal events,
healthy and thus unlikely to reproduce); then the S r least and for each elimination-dispersal event each bacterium in
healthy bacteria die and the other S r healthiest bacteria each the population is subjected to elimination-dispersal with
split into two bacteria, which are placed at the same location. probability ped . We assume that the frequency of
Other fractions or approaches could be used in place of (1); chemotactic steps is greater than the frequency of repro-
this method rewards bacteria that have encountered a lot of duction steps, which is in turn greater in frequency than
2
2
concentration function (both a type of landscape), selection 15 15
and bacterial reproduction (bacteria in the most favorable 10 10
environments gain a selective advantage for reproduction), 5 5
crossover and bacterial splitting (the children are at the 0 0
0 10 20 30 0 10 20 30
same concentration, whereas with crossover they generally 1 1
end up in a region around their parents on the fitness land- (a) (b)
scape), and mutation and elimination and dispersal. How- 30 30
ever, the algorithms are certainly not equivalent, and 25 25
neither is a special case of the other. Each has its own distin- 20 20
2
2
guishing features. The fitness function and nutrient concen- 15 15
tration functions are not the same (one represents 10 10
likelihood of survival for given phenotypic characteristics, 5 5
0 0
whereas the other represents nutrient/noxious substance 0 10 20 30 0 10 20 30
concentrations, or perhaps other environmental influences 1 1
such as heat or light). Crossover represents mating and re- (c) (d)
sulting differences in offspring, something we ignore in the
Figure 5. Bacterial motion trajectories, generations 1-4, on
bacterial foraging algorithm (we could, however, have made contour plots. (a) Generation 1, (b) generation 2, (c) generation 3,
less than perfect copies of the bacteria to represent their and (d) generation 4.
splitting). Moreover, mutation represents gene mutation
and the resulting phenotypical changes, not physical dis-
persal in a geographical area. From one perspective, note 30 30
that all the typical features of genetic algorithms could aug- 25 25
ment the bacterial foraging algorithm by representing evo- 20 20
2
2
2
2
15 15 near the global minimum, after one reproduction step, all the
10 10 bacteria are close to it (and remain this way). In this way, the
5 5 bacterial population has found the global minimum.
0 0
0 10 20 30 0 10 20 30
1 1
Swarming Effects
(c) (d)
Here we use the parameters defined earlier to define the
Figure 7. Swarm behavior of E. coli on a test function. (a) cell-to-cell attraction function. Also, we choose S = 50,
Generation 1, (b) generation 2, (c) generation 3, and (d) generation 4. N c = 100, N s = 4, N re = 4, N ed = 1, ped = 0.25, and the C (i ) = 01,
.
i = 1,2,K , S . We will first consider swarming effects on the nu-
work along these lines) and then proceed to consider forag- trient concentration function with the contour map shown
ing algorithms in this context. Finally, note that evolution on Fig. 7, which has a zero value at [15 ,15]T and decreases to
designed the bacteria to forage in a time-varying and noisy successively more negative values as you move away from
environment (i.e., to achieve robust optimization for noisy that point; hence, the cells should tend to swim away from
time-varying cost functions). Can we exploit the character- the peak. We will initialize the bacterial positions by placing
istics of such an optimization approach for engineering all the cells at the peak [15 ,15]T . Using these conditions, we
problems? get the result in Fig. 7. Notice that in the first generation, the
cells swim radially outward, and then in the second and
Example: Function Optimization via third generations, swarms are formed in a concentric pat-
Bacterial Foraging tern of groups. Notice also that with our simple method of
As a simple illustrative example, we use the algorithm to try simulating health of the bacteria and reproduction, some of
to find the minimum of the function in Fig. 3 (note that the the swarms are destroyed by the fourth generation. We omit
point [15 ,5]T is the global minimum point). To gain more in- additional simulations that show the behavior of the swarm
sight into the operation of the algorithm, it is recommended on the surface in Fig. 3 since qualitatively the behavior is as
that you run the algorithm yourself (see the sidebar on p. 61 one would expect from the above simulations. The inter-
for a Web address from which you can obtain the code). ested reader can obtain the code mentioned above and fur-
ther study the behavior of the algorithm. Note, however,
Nutrient Hill-Climbing: No Swarming that simulation of swarming mechanisms is somewhat deli-
According to the above guidelines, choose S = 50, N c = 100, cate and stretches the simple inaccurate model that we are
N s = 4 (a biologically motivated choice), N re = 4, N ed = 2, using for the bacteria.
ped = 0.25, and the C (i ) = 01,
. i = 1,2,K , S . The bacteria are ini-
tially spread randomly over the optimization domain. The
Biomimicry of Foraging for Control:
results of the simulation are illustrated by motion trajecto-
ries of the bacteria on the contour plot of Fig. 3, as shown in
Challenges and Directions
Fig. 5. In the first generation, starting from their random ini- It is certainly impossible to explore all the potential uses of
tial positions, searching occurred in many parts of the opti- foraging algorithms in this single article, even if we only fo-
mization domain, and you can see the chemotactic motions cused on the field of control. To conclude this article, how-
of the bacteria as the black trajectories where the peaks are ever, we next point to some ideas on the potential uses of
avoided and the valleys are pursued. Reproduction picks foraging algorithms in control to give the reader a flavor of
the 25 healthiest bacteria and copies them, and then as their potential applicability. Even from this brief discussion,
shown in Fig. 5 in generation 2, all the chemotactic steps are it should seem at least plausible that there are applications
in five local minima. This again happens in going to genera- of the methods to optimization, optimal control, model pre-
tions 3 and 4, but bacteria die in some of the local minima dictive control, adaptive estimation and control, and com-
(due essentially to our requirement that the population size puter-aided control system design.
dh( t ) d 2gh( t ) c
= + u( t )
dt A( h( t )) A( h( t )) r(t) u(t) y(t)
Controller Plant
proaches? For example, could we get better tracking perfor- one the animal is found in and hence optimized for via evolu-
mance using other methods (or by tuning this one better)? tion? We need to understand the social foraging strategies of
Which approach is easier to develop so that it can achieve several organisms, try to understand why these are good for
acceptable performance? Which is more computationally the environment they live in, and then seek to develop anal-
complex and more difficult to implement? Can we mathe- ogous implementable strategies for groups of UAVs. Second,
matically prove that the tracking error will converge? What it would be interesting to characterize the physiological and
class of problems might be particularly appropriate for this environmental aspects that drove evolution to design a
type of approach? While we did not illustrate it above, com- specific foraging strategy and optimize its operation. This
pared to standard multiple-model adaptive control ap- would help us understand how vehicular constraints and
proaches, the social foraging approach suggests ideas for tactical situations affect the design and operation of the co-
sharing information between different parameter update operative strategy. Perhaps it would also suggest ideas for
mechanisms for different identifier models (as do genetic how to optimize the design of cooperative control strategies
adaptive control methods). How fruitful will such ideas be for UAVs.
for applications? It is beyond the scope of this article to ad- The warfare strategies (foraging) employed by many ani-
dress these questions. mals have been optimized via evolution for millions of
years; it seems logical that they may suggest some novel ap-
Autonomous Vehicle Guidance: proaches to the design of guidance strategies for UAVs. Can
What Can Nature Teach Us? we find engineering applications that call for functionalities
The artificial potential field method in autonomous vehi- that are similar to what evolution has fine-tuned for an or-
cle guidance bears some similarities to bacterial foraging al- ganism, then exploit the bio-design for solution to practical
gorithms. There are clear analogies between foraging and technological problems? Or, put another way, can we find bi-
cooperative control of groups of uninhabited autonomous ological systems that can solve technological problems that
vehicles (UAVs) that are used in military (or commercial) are beyond our current engineering capabilities?
applications: i) Animals, organisms = UAVs, ii) social forag-
ers = group of cooperating UAVs that can communicate with
each other, iii) prey, nutrients = targets, iv) predators, nox- The author would like to thank Prof. H. Berg of Harvard Uni-
ious substances = threats, and v) environment = battlefield. versity for his helpful comments on the description of the
Are these analogies useful? Biomimicry of social foraging of motile behavior of E. coli bacteria. The author would also
ants [3] has provided some concepts for UAV problems [35], like to thank the reviewers and editors for helpful sugges-
[36]. What future research directions does this article sug- tions on how to improve the article. The author would like to
gest along these lines? Some are as follows: First, the utility thank DAGSI/AFRL for financial support.
of further development of the social foraging metaphor
needs more study. For example, do bacteria employ cooper- References
ative control methods that would also be effective strate- [1] D. Stephens and J. Krebs, Foraging Theory. Princeton, NJ: Princeton Univ.
gies for military applications? Do other social foraging Press, 1986.
animals? For example, ones that operate with a similar [2] W. OBrien, H. Browman, and B. Evans, Search strategies of foraging ani-
physiology as a UAV and in a similar environment to the mals, Amer. Scientist, vol. 78, pp. 152-160, Mar./Apr. 1990.