Journal of Intelligent and Robotic Systems 3: 51-66, 1990.

51
1990 Kluwer Academic Publishers. Printed in the Netherlands.
Neural Networks in Robotics: A Survey*
BILL HORNE, M. JAMSHIDI and NADER VADIEE
CAD Laboratory for Systems~Robotics, Department of Electrical and Computer Engineering,
University of New Mexico, Albuquerque, NM 87131, U.S.A.
(Received: 12 December 1988; revised: 5 March 1989)
Abstract. The purpose of this paper is to provide an overview of the research being done in neural network
approaches to robotics, outline the strengths and weaknesses of current approaches, and predict future
trends in this area.
Key words. Neural networks, robotics, nonlinear control.
1. I nt roduct i on
Neur al net wor ki ng has become one of the mos t popul ar t opi cs in the scientific
communi t y wi t hi n t he past few years, mar ked by record conference at t endances and
publ i cat i ons. Ma ny disciplines in science have been affect ed by this ' mi ni - r evol ut i on' ,
and r obot i cs is no except i on. Thi s is i l l ust rat ed in the fact t hat eighty-five per cent of
the paper s revi ewed in this article have been publ i shed wi t hi n the past t hree years. The
paper s revi ewed in this paper pr i mar i l y reflect appr oaches t o r obot i cs whi ch ma y have
near t er m appl i cat i on t o i ndust ri al robot i cs. Mos t of this research is not commi t t ed
to expl ai ni ng physi ol ogi cal aspect s of huma n mo t o r cont rol . However , t here is
anot her l arge body of publ i cat i on whi ch is commi t t ed t o this goal (for exampl e, see
Bul l ock and Gr os s ber g [7]).
Thi s paper is st r uct ur ed as follows. Section 2 pr ovi des a br i ef overvi ew of neural
net wor ks, some of the criticisms whi ch arise concer ni ng t he rol e of neural net wor ks
in robot i cs, and a descri pt i on of the br oom- bal anci ng pr obl em as a hi st ori cal per-
spective. Sect i on 3 provi des a br i ef out l i ne of robot i cs. Section 4 summar i zes specific
research in neural net wor k appr oaches t o robot i cs. Section 5 pr ovi des a s umma r y and
critical anal ysi s of the appr oaches out l i ned in this paper .
2. An Overvi ew o f Ne ur al Ne t wo r ks
Neur al net wor ks are syst ems whi ch have been deri ved t hr ough model s of neur ophysi -
ol ogy. I n general , t hey consi st of a col l ect i on of si mpl e nonl i near comput i ng el ement s
whose i nput s and out put s are tied t oget her to f or m a net wor k. The mai n advant ages
of neural net wor ks for r obot i cs is t hei r ability to adapt i vel y l earn nonl i near funct i ons
whose anal yt i c f or ms are difficult t o deri ve and whose sol ut i ons are har d to comput e.
* This work was supported, in part, by Sandia National Laboratories under contract No. 06-1977,
Albuquerque, New Mexico.
52 BI LL HOR NE ET AL.
The most domi nant forms of neural net works used in robot i cs are the mul t i -l ayer
per cept r on and t he Hopfi el d net work. We will give a br i ef descri pt i on bel ow of the
mechani cs of these net works for t hose who are not familiar with them. However,
many ot her net works have been pr oposed f or robot i cs including Compet i t i ve and
Cooper at i ve nets [4], Rewar d/ Puni shment nets [6], etc. Wher e appr opr i at e, sources for
mor e i nf or mat i on on these net works have been referenced.
2.1. THE MUL T I - L AYE R P E R C E P T R ON
The basic comput at i onal el ement in a mul t i -l ayer per cept r on ( MLP) is the per cept r on
[37], shown in Fi gure 1. A per cept r on receives N cont i nuousl y valued inputs, x~,
i = 1, 2, . . . , n, f r om either an external source (e.g. a sensor) or ot her percept rons.
The inputs are multiplied by a scalar value which is t hen added to a bias weight to
f or m the i nt ermedi at e value y. A nonl i near funct i on is t hen applied to y to f or m the
out put of the percept ron, u.
Some typical nonl i near funct i ons are shown in Fi gure 2. One of these funct i ons is
a hard-limiter. When this funct i on is appl i ed to the weighted sum, the per cept r on
effectively forms a linear decision region in the i nput space. I f percept rons are l ayered
as shown in Fi gure 3, t hen mor e compl ex decision regions can be formed. Multi-
layered net works can f or m ar bi t r ar y decision regions including non- convex and
mul t i -modal regions [17, 24]. We woul d like the MLP to achieve these decision
regions adapt i vel y by example, unf or t unat el y t here is no learning al gori t hm f or
MLPs when hard-l i mi t ers are used. Instead, a funct i on is requi red which is mono-
tonically increasing and differentiable. Such a funct i on is a sigmoid which is also
shown in Fi gure 2. The resulting weight updat e is f or med by t aki ng the gradi ent of
the t ot al squared er r or (squared difference between actual out put and desired out put
for a t rai ni ng sample) with respect t o the weights and per f or mi ng a gradi ent search
of the weight space. The resulting learning al gori t hm is called b a c k - p r o p a g a t i o n
because the errors are pr opagat ed backwards t hr ough the net work. The equat i ons are
given by [38]:
w i j , k ( n + 1) = w i j , k ( n ) + p e i , k ui , k ( 1 - - ui, k ) u i I j
X
1

2
W W o
O
Fig. 1. A per cept r on.
NEURAL NETWORKS IN ROBOTICS: A SURVEY 53
1.0
0.8
0.6
~4~ 0 . 4
0.2
0.0
/ 9~-hard limit~r
I
- 2 0 2
X
Fig. 2. Typical non-linearities.
OUTPUTS
INPUTS
Fig. 3. A multi-layered perceptron.
wher e
Nj
e i . k = y~ ui +, . , , (l - U , + , . m ) e i + l , , , , W i + l , k ....
m - I
e i , k is t he er r or t er m f or k t h node in i t h l ayer, Ui,k t he o u t p u t o f kt h node in i t h l ayer,
w < j . k t he wei ght c onne c t i ng t h e j t h node in l ayer (i - 1) t o t he k t h node in l ayer i, y
t he conver gence rat e, and Ni t he n u mb e r o f nodes in i t h l ayer.
Thi s t ype o f ne t wor k is still useful f or cl assi fi cat i on pr obl ems . Howe ve r , since t he
out put o f t he net wor k has c ont i nuous val ues in t he r ange (0, 1), t he ML P can be used
54 BILL HORNE ET AL,
to f or m a cont i nuous nonl i near mappi ng. Not e here t hat classification is consi dered
j ust a special case of a nonl i near mappi ng. This is the pr oper t y most useful for
robot i cs since kinematics (forward and inverse), dynamics, sensing and cont r ol can all
be described in terms of nonl i near mappings.
Ther e are a number of probl ems with using MLPs. First, t here is no guarant ee t hat
an MLP will converge to a local mi ni mum within the weight space since the gradi ent
used in the weight updat e is the gradi ent f or a given t rai ni ng sample and may not
represent the overall gradient. A t echni que called bat chi ng can be used to comput e an
er r or based on many samples t hus giving a mor e realistic estimate of the gradient.
However in practice we have f ound t hat this t echni que does not provi de any signifi-
cant i mpr ovement over using an i nst ant aneous estimate of the gradient. Second, the
MLP i mpl ement s onl y an appr oxi mat i on to an act ual nonl i near mappi ng. The
accuracy of this appr oxi mat i on may be quest i onabl e. Thi rd, t here is no known
met hod for det ermi ni ng the number of nodes, convergence paramet ers, etc., for a
given probl em. Finally, t here is no known way t o ' pr epr ogr am' a pri ori knowl edge
into t he net work. As a result the MLP starts in a compl et el y r andom state and may
require a long training peri od to converge.
2.2. THE HOPFIELD NETWORK
The Hopfield network [44] has many variations, the simplest of which is discussed
here. Hopfi el d net works are used f or associative memor y and combi nat or i al opti-
mi zat i on. The latter pr oper t y has been the most frequent l y used for robot i cs applica-
tions. The Hopfi el d net wor k is a one-l ayer net wor k with feedback connect ed in a
crossbar t opol ogy as shown in Fi gure 4. Each node uses a hard-l i mi t i ng nonl i neari t y
(vari at i ons use ot her nonlinearities). The equat i on of the out put is given by:
u ( k + 1) = f ( T u ( k ) + x) .
An energy funct i on for the net wor k can be defined as:
E - - l b l T T b l - - u T x , ,
U . U 2 o o o U
X I
2
e
o
X n
"-|
t~n|
~ 2
t'~n2
O- - -
o.o-
0
0
Fig. 4. The Hopfield network.
NE UR AL NE T WOR KS I N ROBOTI CS: A S URVEY 55
where u is the out put vect or of net work, x the i nput vect or of net work, T t he mat ri x
of i nt er connect i on weights, f the nonl i neari t y (e.g. hard-limiter), and E the energy.
Concept ual l y, the out put updat e equat i on serves to per f or m a mi ni mi zat i on of t he
energy funct i on. * This pr oper t y can be used f or solving combi nat or i al opt i mi zat i on
probl ems [44, 52]. An exampl e is the travelling salesman pr obl em in which a salesman
must visit N cities and minimize t he distance travelled over the entire t our. The onl y
i nf or mat i on given is t he distance bet ween pairs of cities. To solve this pr obl em a~
energy funct i on is defined so t hat the mi ni mi zat i on of this funct i on cor r esponds to a
sol ut i on of the probl em.
An advant age of the Hopfi el d net wor k is t hat t here exists a pr oof of convergence
based on Lyapunov analysis [44, 52] and, f ur t her mor e, the convergence rat e
is i ndependent of the number of nodes. However , even t hough the net wor k is
guar ant eed to converge, it oft en converges to local mi ni ma which are undesirable. For
example, in the travelling salesman probl em, solutions are oft en reached which do not
satisfy const rai nt s of t he probl em, e.g. the salesman does not visit every city. In the
travelling salesman pr obl em the energy funct i on can be easily defined since t he
s ol ut i on onl y needs to meet a limited number of constraints. In mor e difficult
probl ems with large number s of const rai nt s it seems likely t hat the net wor k will be
even mor e likely to arrive at invalid solutions.
2.3. A HI S T ORI CAL PERSPECTI VE: THE ' B R OOM- B AL ANC I NG' P ROBL E M
The br oom- bal anci ng pr obl em (also called the i nvert ed pendul um or car t - pol e
system) is a classic exampl e of the appl i cat i on of neural net works to cont rol . The
br oom- bal anci ng pr obl em consists of an i nvert ed pendul um of length L a nd mass m
mount ed on a cart of mass M, as shown in Fi gure 5. The goal of the cont r ol l er is to
keep the pol e bal anced (i.e. the angle 0 = 0) and mai nt ai n the cart at its origin (i.e.
X = 0) by appl i ng a force, F, in the hor i zont al direction.
The equat i ons of mot i on governi ng the system are:
3
0 = 4-~ (g sin 0 - ] ( c o s 0)
and
m(L sin 002 - -~g sin 20) - FX + u
2=
M + m(1 - cos 2 0)
This is an undamped and i nherent l y unst abl e f our t h- or der system whose dynami cs are
nonl i near and coupl ed. The dynami cs of many cont r ol probl ems including r obot i c
cont r ol are, in general, nonl i near and coupl ed. Ther ef or e the br oom- bal anci ng pro-
blem is oft en consi dered a pr oof of principle of neural net works in robot i cs.
* Not e t ha t a l t hough t hi s ener gy f unc t i on has a qua dr a t i c f or m, u ha s a non- l i near f or m (t ypi cal l y
non- pol ynomi a l ) and t hus t he ener gy f unct i on can have mul t i pl e mi ni ma .
56 BILL HORNE ET AL.
0 4
e
i
v
Vl
X
Fig. 5. The cart-pole system.
Originally, the br oom- bal anci ng pr obl em was solved by Wi dr ow and Smith
[42, 50, 51]. Not onl y was this the first appl i cat i on of neural net works to cont rol ,
but it was one of the first appl i cat i ons of neural net works to any probl em.
Wi dr ow and Smith used ' bang-bang' cont r ol (i.e. the cont r ol l er can appl y an
impulsive force F of fixed magni t ude in ei t her the positive or negative X di rect i on)
with a single adapt i ve linear el ement ( ADALI NE) . This device was t rai ned
by observi ng a human t eacher operat e t he system manual l y. Fur t her mor e,
Wi dr ow and Smith physically i mpl ement ed t he system as opposed t o simulation.
Mor e recently Tol at and Wi dr ow [45] have si mul at ed this system using visual
inputs only. Ther e are two issues to consi der about this approach: (i) bang- bang
cont r ol is essentially a classification probl em, and (ii) the net wor k l earned f r om
a human teacher. Bot h of these condi t i ons will not , in general, ext end to r obot i c
cont rol .
Guez and Selinsky [11, 12] have appl i ed MLPs to this pr obl em as a simulation.
In doi ng so the cont r ol forces appl i ed to the cart are cont i nuous values. One
interesting out come of this research is t hat it was observed t hat the per f or m-
ance of t he net wor k was much bet t er t han t hat of the human teacher. This
i mpr ovement is due to the fact t hat human cont r ol is oft en inconsistent and
t hat t here are physiological limitations in humans which the neural net wor k can
overcome.
Bart o et al. [6] and Ander son [5] have simulated rei nforcement learning f or the
br oom- bal anci ng probl em. Like Wi dr ow and Smith these systems use bang- bang
cont rol . Her e the state space is quant i zed using a decoder. A neuron-l i ke el ement
called an Adapt i ve Search El ement (ASE) per f or ms the classification. The reinforce-
ment is ai ded by an Adapt i ve Critic El ement (ACE). The pur pose of the ACE is to
provi de an el ement of predi ct i on as t o what t he rei nforcement shoul d be. A signal,
NE UR AL NE T WOR KS I N ROBOTI CS: A S URVEY 57
r, is supplied to the network which reinforces network weights if the pole remains
balanced and "punishes" weights which result in the pole falling past some nominal
value of 0.
An important issue is that the work of Widrow, Barto, and colleagues requires
quantization of the input space with a decoder and thus resembles table look-up
methods. For the broom-balancing problem the level of granularity can be fairly
coarse and still yield good performance. However, in general most problems will
require finer quantization. A study of Raibert [35] indicates that this approach is, in
general, impractical for robotic control. The multi-layer perceptron approach used by
Guez and Selinsky incorporates the decoding functionality implicitly in the network
and thus avoids this problem. One way of thinking about this approach is that a
three-layered perceptron can account for the functionality of a decoder and a single
classification element combined.
3. An Overview of Robotics
Robotics constitutes the study of a finite number of rigid mechanical chains which
represent a multi-variable non-linear coupled system. The solution of this problem is
difficult because even the simplest desired movement requires sophisticated and
computationally intensive mathematics. Major problem areas of robotics include
kinematics, dynamics, trajectory planning, sensing, control, computer languages and
intelligence.
3.1. KI NE MAT I CS
Kinematics refers to the study of robot joint motions without considering the causes
of motion. Two distinct subproblems are distinguished here. The forward kinematics
problem involves a non-linear matrix mapping from joint space (i.e. a description
of the robot in terms of joint angles/positions) of the robot to Cartesian space
(i.e. Cartesian coordinates of the robot end-effector). The forward kinematic solution
computes the Cartesian location of the end-effector given the joint space descrip-
tion. The computation is relatively straight forward but requires several non-linear
trigonometric and matrix multiplications. To do this independent coordinate systems
are associated with each link of the robot. A point, p~, expressed with respect to
link i can be expressed with respect to the coordinate system in link i - 1 by the
equation
Pi = i - l A i P i ,
where i-~A i is the Denavit-Hartenberg transformation matrix. The inverse problem
is, on the other hand, a somewhat more difficult task. The inverse kinematic solution
computes the joint space coordinates given the end-effector location in Cartesian
58 BILL HORNE ET AL.
coordinates. This probl em can yield a multiple number of solutions. In general, the
inverse kinematic solution is more comput at i onal l y intensive t han the forward
kinematic solution.
3.2. DYNAMICS
Robot arm dynami cs f or mul at es the mappi ng between j oi nt torques applied to the
robot and the j oi nt coordinates, velocities and accelerations. Robot arm dynami cs is
difficult because it requires i ncorporat i ng effects of inertia, coupling between j oi nt s
(Coriolis and centrifugal), gravity loading, and potentially backlash, gear friction, and
the dynami cs of the control devices. Solutions such as the Lagr ange- Eul er (L-E)
formul at i on and the Newt on- Eul er (N-E) formul at i on require large numbers of
t ri gonomet ri c and nonl i near functions of the j oi nt coordinates, velocities and
accelerations. For example, the Lagrange-Eul er equat i on for a simple two link
mani pul at or with rot ary joints, and equal length links, is found to be
Zl = 3 , + ~m2 I2 + m2C2 I2 m2 I2 + m212C2 "01
l m 1 2 m212C2 l m212 "02
"E2 ~ 2* +
where z i is the t orque applied to j oi nt i to drive link i, mi is the mass of the i t h link,
l the length of each link (assumed equal), g the gravitational const ant , C~ the cos (0~),
S~ the sin (Oi), and C,s the cos (Oi + Oj).
3.3. TRAJECTORY PLANNING
Traj ect ory pl anni ng is the process of defining a desired pat h in j oi nt or Cartesian space
for j oi nt s or gripper, respectively. Generally this requires comput i ng a set of poly-
nomi al functions to generate a sequence of desired reference points. The probl em is
further complicated when the robot must move ar ound obstacles or is constrained to
a given path.
3.4. SENSING
The use of external sensing mechanisms allows the robot to interact with its environ-
ment in a more efficient manner. Types of sensors include position, velocity, accelera-
tion, range, proximity, force, tactile, and visual. However, the performance of sensing
systems is relatively primitive. Most industrial robot s use minimal sensory feedback.
3.5. CONTROL
Given a dynami cal description of a robot the purpose of the control modul e is
to mai nt ai n the dynami c response of the mani pul at or in accordance with some
NE UR AL NE T WOR KS I N ROBOTI CS: A S URVEY 59
prespecified desired trajectory. In general control is difficult because the dynamics of
the links in a robot arm are nonlinear and coupled. Control techniques include
computed torque control [26], and resolved motion rate control [48], A major problem
is controlling a robot which handles large payloads or works in contact with the
environment since the dynamics of the robot changes as force is imposed by the
environment. Control techniques to deal with this problem include damping control
[49], hybrid position/force control [36], and impedance control [16].
3.6. TAS K P L ANNI NG AND I NT E L L I GE NCE
Finally, other important areas in robotics are task planning and intelligence. It is
desirable to give the robot a high-level task to perform and have the robot develop
an intelligent plan to perform the task. These problems involve the coordination of
multiple sources of data and require problem-solving techniques. Also more flexible
robotics systems will have to be developed to be able to operate in a changing and
harsh environment. When robots are inflexible a great deal of time must be invested
in setting up a friendly work space and environment. Robots will have to be equipped
with some sort of intelligence to deal with uncertainties in the environment.
4. Summary of Neural Network Approaches to Robotics
4.1. NE UR AL NE T WOR KS I N KI NE MAT I CS
The inverse kinematics (IK) problem, as discussed before, is a difficult one because it
is computationally intensive and has multiple solutions. Neural networks may be able
to reduce the computational complexity of the inverse kinematics problem. However,
analytic [K solutions yield numerically accurate results whereas the neural network
solution in general may not.
Iberall [18, 19] has used cooperative/competitive neural networks [4] to compute the
inverse kinematic solution for fingers in a simulated robot hand during grasping.
Guez and Ahmad [10] have investigated inverse kinematic solutions for two- and
three-degree-of-freedom manipulators using MLPs. Because of the relative numerical
accuracy of neural networks compared with closed form solutions, Guez and Ahmad
suggest that neural networks may be best as a good initial estimate for a manipulator
that requires iterative methods for its solution.
4.2. NE UR AL NE T WOR KS I N DYNAMI CS
The nonlinear mapping property of neural networks is ideal for robot dynamics. The
basic idea is that the neural network learns the inverse dynamical relationship of the
robot directly which can be used as a inverse dynamics controller. Miyamoto e t al . [30]
and Kawato e t al . [21, 22] have implemented a novel neural network for inverse
dynamics control based on neurophysiology. They basically used a single perceptron
60 BILL HORNE ET AL.
which receives values f r om multiple fixed non-linearities of the i nput variables. The
aut hor s const rai n the system by requi ri ng a p r i o r i knowl edge of t h e f o r m of the inverse
dynami cs f or mul at i on (i.e. Lagr ange- Eul er ) . Thi s pr oper t y may be useful in esti-
mat i ng coefficients of the L- E equat i ons of an unmodel l ed r obot . In this sense, the
aut hor s' appr oach resembles system identification techniques. Thei r appr oach is
i nherent l y parallel and adapt i ve. The aut hor s have i mpl ement ed t he net wor k on a
Hewl et t - Packar d 9600-300-320 f or a PUMA 260 mani pul at or .
One issue is how the r obot learns the inverse dynami cs mappi ng. Psaltis e t al .
[33, 34] provi de an excellent overview of different learning techniques. One pr obl em
t hey discuss is how t o effectively t rai n the neural net wor k on-line wi t hout using a
r andom cont r ol regime. An interesting pr oposal involves back- pr opagat i ng errors
t hr ough the r obot to updat e the neural net. However , this requires some a p r i o r i
estimate of the dynami cs of the r obot and the sol ut i on will have errors related to the
er r or in this estimate.
4.3. NEURAL NETWORKS IN TRAJECTORY PLANNING
Jorgenson [20] has investigated t he use of si mul at ed anneal i ng [38] for mobi l e r obot
pat h planning. However , he not es t hat simulated anneal i ng is t oo comput at i onal l y
intensive f or real-time appl i cat i ons such as robot i cs. Obstacle avoi dance in an
unknown envi r onment has been i mpl ement ed by Tsut sumi e t al . [46, 47] using
Hopfi el d nets for mul t i -j oi nt r obot s and truss structures. Here, whenever a new
obstacle is found, the weights are updat ed by addi ng a t erm t o t he energy funct i on.
The energy funct i on used to describe the net wor k weights is quite compl ex and may
be difficult to i mpl ement in practice. Seshadri [39] has investigated the use of Hopfi el d
nets for mobi l e r obot pat h planning. His f or mul at i on resembles the travelling sales-
man pr obl em in t hat t he purpose of the neural net wor k is t o minimize the length of
a pat h t o a goal position. Her e the system is const rai ned when obstacles are in the
wor k space. Al t hough Seshadri does not give the energy funct i ons, it seems his
appr oach woul d be simpler t han t hat of Tsut sumi e t al . described above. However ,
the t radi t i onal f or mul at i on of the travelling salesman pr obl em is equally effective f or
this task, and since the weights of the net wor k are fixed, the system cannot deal with
unant i ci pat ed obstacles.
Li u e t al . [25] have used a MLP f or classifying vari ous r obot i c hand grips based on
characteristics of t he object to be grasped. This wor k is an excellent exampl e of how
classification propert i es of neural net works can be useful in robot i cs.
Eckmiller [8] has devel oped a novel neural net wor k called a neural t ri angul ar lattice
( NTL) for st ori ng and retrieving trajectories.
4.4. NEURAL NETWORKS IN SENSING
Al t hough the use of neural net works in vision is qui t e extensive, a discussion of this
research is beyond the scope of this paper. The onl y ot her wor k we have come across
NEURAL NETWORKS IN ROBOTICS: A SURVEY 61
in sensing is t hat of Pati et al. [32] who investigate the use of neural net works in tactile
percept i on. The basic inversion pr obl em of tactile percept i on is a deconvol ut i on t o
recover the surface stress f r om strain dat a. Pati use the deconvol ut i on propert i es of
a Hopfi el d net [27] t o per f or m the mappi ng.
4.5. NEURAL NETWORKS IN CONTROL
The bul k of neural net wor k research for robot i cs has been done in cont rol . Inverse
dynami cs cont r ol was discussed in Section 4.2. Ot her appr oaches are diverse and use
many different types of net works.
Pao and Sobajic [31] and Sobajic et al. [43] i mpl ement ed a posi t i onal cont r ol
al gori t hm f or a t wo-degree-of-freedom mani pul at or using an iterative I K solution.
The aut hor s have i mpl ement ed the system using a MLP r unni ng on an I BM PC/ AT
t o cont r ol an Intelledex 605T mani pul at or .
Al bus [1, 2, 3] presents his Cerebel l ar Model Art i cul at i on Cont r ol l er ( CMAC)
for general r obot i c cont rol . This appr oach can be described as a di st ri but ed
t abl e-l ookup met hod. Most of his papers are concerned wi t h the propert i es of
CMAC and do not discuss cont r ol issues at any great length. Miller [29] has
i ncor por at ed CMAC i nt o a comput ed t or que cont r ol l er and has at t ai ned impressive
si mul at i on results.
Guez et al. [13, 14, 15] simulate Hopfi el d net works in Model Reference Adapt -
ive Cont r ol ( MRAC) . Her e the net wor k adjusted the paramet ers of the MRAC
cont rol l er as opposed to provi di ng a compl et e i nput / out put mappi ng of a
cont rol l er.
Elsley [9] has simulated an inverse Jacobi an cont r ol l er using MLPs. Elsley claims
bet t er per f or mance on l ong movement s t han a t radi t i onal inverse Jacobi an cont r ol l er
typically provides. Shepanski and Macy [40, 41] simulate aut omobi l e driving using
MLPs. They discuss the rel at i onshi p bet ween t hei r appr oach and an expert system
appr oach.
Kuper st ei n [23] uses a novel neural net wor k called I NF ANT (Int eract i ng Net wor ks
Funct i oni ng on Adapt i ve Neur al Topol ogi es) for posi t i onal cont r ol using visual
inputs. Thi s system is highly mot i vat ed by neurol ogi cal and devel opment al dat a.
Unf or t unat el y Kuper st ei n does not give much detail on the pur e robot i cs aspects of
the system.
4.6. NEURAL NETWORKS IN TASK PLANNING AND INTELLIGENCE
Albus [2] has pr oposed t hat intelligent cont r ol coul d be i mpl ement ed with a hi erarchy
of CMAC modules. At each successive level of the CMAC hi erarchy, a CMAC
modul e woul d decompose a high level command into a set of l ower level commands.
A command such as ' Build a widget' coul d be given t o the CMAC hi erarchy which
woul d decompose the sol ut i on i nt o successively simpler commands at each level until
62 BILL HORNE ET AL.
the lowest level CMAC finally comput es the requi red act uat or t orques. Ot her wor k
related to intelligent robot i cs includes a pr oposal f or cont rol l i ng multiple r obot s
si mul t aneousl y [53], f or i ndust ri al manufact uri ng.
5. Summary and Analysis
5.1. CRITICISMS AND ISSUES CONCERNING NEURAL NETWORKS IN ROBOTICS
This section focuses on some of the criticisms which face neural net wor k cont r ol
paradigms. Much of the research discussed t hr oughout this paper addresses these
issues directly.
Neural networks are primarily classification networks. Many peopl e feel t hat neural
net works (pri mari l y MLPs) are best appl i ed to cont r ol probl ems where t here is some
t ype of classification being performed. It is cert ai nl y t rue t hat some very interesting
cont r ol probl ems have been i mpl ement ed using the classification abilities of MLPs.
For example, t hey can be used in relay or bang- bang cont r ol systems. However , as we
have discussed classification in these net works is j ust one type of non-l i near mappi ng.
The most promi si ng pr oper t y of neural net works is their ability to adapt i vel y learn
compl ex mappings. One superficial advant age of this pr oper t y is t hat it allows us to
avoi d deriving some closed f or m anal yt i c funct i on by hand. But mor e i mpor t ant l y, the
system coul d l earn mappi ngs which are mat hemat i cal l y intractable. In addi t i on, the
system woul d be por t abl e since it adapt s to the r obot (or envi ronment ) t o which it is
applied.
Neural networks must learn f r om a teacher. The criticism here is t hat neural net-
works must learn to i mi t at e some t ype of convent i onal cont rol l er. But this is of little
use since the convent i onal cont r ol l er is al ready available and t he neural net wor k
cannot i mprove upon its performance. However, many net works l earn f r om a con-
t rol l er whi ch has no anal yt i c model , e.g. a human teacher. Perhaps the most powerful
pr oper t y of neural net works is t hei r ability to model the cont rol l ed system itself. In
robot i cs, f or example, it may be possible to i mpl ement a compl et e inverse dynami cs
model of the r obot which coul d possibly i ncor por at e dynami cs of the cont r ol device,
backlash, and gear friction. Thi s model woul d be comput ed wi t hout the need for
anal yt i c modelling.
Al t hough some wor k has been done with r ei nf or cement learning [5, 6], it is not clear
how to appl y these principles to general r obot i c cont rol .
Speed considerations: Simulation vs Implementation. Most neural net works cannot
run in real time. Al t hough neural net works are i nherent l y parallel no har dwar e exists
to i mpl ement t hem in parallel. They do not map nicely into existing digital parallel
archi t ect ures such as hypercubes, t r ansput er s, or the Connect i on Machi ne. Thi s is
because neural net works requi re simple processing but compl ex i nt er communi cat i on
requirements. In most existing parallel architectures, i nt erprocess communi cat i on
tends to be biggest bot t l eneck. Opt i cal i mpl ement at i ons which will run in real time are
several years away.
NE UR AL NE T WOR KS I N ROBOTI CS: A S URVEY 63
Thr oughout t he paper we have been careful to describe whet her the research we have
reviewed is merel y a proposal , a simulation, or an i mpl ement at i on.
On-line vs off-line learning. Rel at ed to the above issue is the quest i on of whet her a
neural net wor k shoul d be t aught on-line or off-line. On-line learning is desirable
because the net wor k woul d pot ent i al l y be able t o adj ust t o changes in the system.
However , due to speed consi derat i ons, it may be necessary to t rai n the net wor k
off-line and use it as a non- adapt i ve system.
Comparisons of neural networks and conventional systems. Perhaps the most
i mpor t ant thing to consi der is whet her a neural net wor k appr oach to robot i cs can do
somet hi ng which a convent i onal appr oach cannot . Thi s is especially i mpor t ant when
consi deri ng t hat a neural net wor k sol ut i on may oft en be numeri cal l y i naccurat e
compar ed to a convent i onal solution. Researchers shoul d either appr oach probl ems
which have no convent i onal sol ut i on or give a compar i son with convent i onal techni-
ques. None of the researches outlined in this paper offered such comparisons.
5.2. P ROBLEM ARE AS I N NE UR AL NE T WOR K AP P ROACHE S TO ROBOTI CS
Pr obl em areas for research in neural net wor k appr oaches to robot i cs can be di vi ded
i nt o f our categories:
(a) Specifying the configuration of the network. Decisions have to be made regard-
ing the type of neural net wor k to be used, its archi t ect ure, t opol ogy, number of layers,
number of nodes, t ype of non-l i neari t y, and associated paramet ers. It is not adequat e
to simply t hr ow a large number of resources at t he probl em. For example, in an MLP
it mi ght seem possible to use mor e nodes t han are needed for a given probl em.
However , it has been observed t hat t oo many nodes results in over-general i zat i on on
the t rai ni ng set and poor per f or mance on new data.
(b) Specifying the teacher. I f supervised learning is adopt ed, t he t ype of t eacher or
expert t hat is available to give the correct out put t o the net has to be chosen. As
previ ousl y discussed, using a convent i onal cont rol l er as the t eacher is useless since the
cont rol l er al ready exists and will out per f or m the neural net. It is not clear how
unsupervi sed learning t echni ques can realize compl ex non-l i near mappings.
(c) Specifying the training set. Th e t rai ni ng set must be specified to give an
adequat e represent at i on of t he t ype of inputs the net wor k is likely to see. We
woul d like to be able to derive some type of opt i mal t rai ni ng set f or specific applica-
t i on domai ns. Oft en r andom t rai ni ng i s used, but this appr oach is i nappr opr i at e f or
on- l i ne l earni ng.
( d) Specifying the learning algorithm. Ther e is a great need f or mor e efficient
learning al gori t hms to speed t rai ni ng and guarant ee convergence. A maj or pr obl em
is t hat as new t rai ni ng samples are learned, old mappi ngs t end to degrade. This results
in the net wor k over-generalizing to recent pat t erns. One sol ut i on to this pr obl em is
to r andomi ze the or der of the t rai ni ng dat a. However , this can onl y be effective in
off-line learning techniques.
64 BILL HORNE ET AL.
5.3. SUMMARY
Research in neural net wor k appr oaches to robot i cs are quite diverse and r udi ment ar y,
leaving much r oom for i mpr ovement and new areas f or devel opment . Much of this
wor k yields poor per f or mance relative t o convent i onal t echni ques part i al l y because
neural net works are not fully under st ood or developed. For example, it is quite
difficult to get an MLP to learn a simple t ri gonomet ri c funct i on much less compl ex
dynami cal f or mul at i ons such as the Lagr ange- Eul er equat i ons.
Hopef ul l y many of these probl ems will di sappear as we devel op and under st and
neural net works mor e fully. Much of the prel i mi nary research discussed here is quite
promising. For example, the fact t hat an MLP can i mpl ement a r obot i c kinematics,
dynamics, or cont r ol mappi ng at all is qui t e r emar kabl e given the convergence
propert i es and relative level of underst andi ng of this net work.
Obviously, the research out l i ned in this paper shoul d not be t aken as the final wor d
in neural net wor k appr oaches to robot i cs. Inst ead most of this wor k shoul d be
consi dered as proof-of-pri nci pl e t hat neural net works can solve some interesting
probl ems in robotics. In the fut ure, as neural net works are i mpr oved and bet t er
under st ood, the solutions to t he probl ems addresses by this paper will be much mor e
compet i t i ve with convent i onal approaches.
Acknowledgements
The aut hor s woul d like, gratefully, to t hank Sandi a Nat i onal Labor at or i es f or sup-
por t i ng this survey paper. We woul d also like to t hank Rebecca Hogenauer for her
val uabl e comment s on the original dr af t of this manuscri pt .
The wor k we have reviewed here is a represent at i ve sample of t he wor k being
done in neural net wor k appr oaches to robot i cs. I f we have missed any part i cul ar
publ i cat i on, we woul d appreci at e receiving a copy for any f ur t her editions of this
paper.
References
I. Albus, J., A new approach to mani pul at or control: the Cerebellar Model Articulation Controller
(CMAC), J. Dynamic Systems, Measurement, and Control, 220-227 (Sept. 1975).
2. Albus, J., Dat a storage in the Cerebellar Model Articulation Controller (CMAC), J. Dynamic Systems,
Measurement, and Control, 228-233 (Sept. 1975).
3. Albus, J., Mechanisms of pl anni ng and probl em solving in the Brain, Mathematical Biosciences 45,
247-293 (1979).
4. Amari, S. and Arbib, M., Competition and cooperat i on in neural nets, In Systems Neuroscience, J.
Metzler (ed.), Academic Press, New York, pp. 119-165 (1977).
5. Anderson, C., Learning to control an inverted pendul um with connectionist networks, I EEE Conf. on
Decision and Control, pp. 2294-2298 (1987).
6. Barto, A. et al., Neuronlike adaptive elements t hat can solve difficult learning cont rol problems, I EEE
Trans. Systems, Man, and Cybernetics, Vol. 13, pp. 834-846 (1983).
7. Bullock, D. and Grossberg, S., Neural dynamics of pl anned arm movements: emergent i nvari ant s and
speed-accuracy properties duri ng trajectory formation, Psychological Review, Vol. 95, No. 1,
pp. 49-90 (1988).
NEURAL NETWORKS IN ROBOTICS: A SURVEY 65
8. Eckmiller, R., Neural network mechanisms for generation and learning of mot or programs, I EEE
Conf. on Neural Net works, Vol. 4, pp. 545-550 (1987).
9. Elsley, R., A learning architecture for cont rol based on back-propagat i on neural networks, I EEE Conf.
on Neural Net works, Vol. II, pp. 584-587 (1988).
10. Guez, A. and Ahmad, Z, Solution to the inverse kinematics problem in robotics by neural networks,
I EEE Conf. on Neural Net works, Vol. II, pp. 617-624 (1988).
11. Guez, A. and Selinsky, J., A trainable neuromorphic controller, J. Robotic Systems 5(4), 363-388 (1988).
12. Guez, A. and Selinsky, J., A neuromorphi c controller with a human teacher, I EEE Conf. on Neural
Net works, Vol. II, pp. 595-602 (1988).
13. Guez, A. et al., Neuromorphi c architecture for adaptive robot control: a preliminary analysis, I EEE
Conf. on Neural Net works, Vol. 4, pp. 567-572 (1987).
14. Guez, A. et al., Neuromorphi c architectures for fast adaptive robot control, I EEE Conf. on Robotics
and Automation, pp, 145-149 (1988).
15. Guez, A. et al., Neural network architecture for control, I EEE Control Syst ems Magazine, Vol. 8,
No. 2, pp. 22-25 (April i988).
16. Hogan, N., Impedance control: an approach to manipulation, J. Dynamic Syst ems, Measurement, and
Control 107, 1-24 (March 1985).
17. Horne, W. - In an unpublished result we have demonst rat ed t hat it is possible to implement a limited
class of multi-modal and non-convex decision regions using a two-layered network, thus countering
Li ppmann' s arguments. For more i nformat i on cont act the author.
18. Iberall, T., A ballpark approach to modelling human prehension, 1EEE Conf. on Neural Net works,
Vol. 4, pp. 535-544 (1987).
19. Iberall, T., A neural network for pl anni ng hand shapes in human prehension, I EEE Conf. on Decision
and Control, pp. 2288-2293 (1987).
20. Jorgenson, C.C., Neural network representation of sensor graphs in aut onomous robot pat h planning,
I EEE Conf. on Neural Net works, Vol. 4, pp. 507-516 (1987).
21. Kawato, M. et al., A hierarchical model for vol unt ary movement and its application to robotics, I EEE
Conf. on Neural Net works, Vol. 4, pp. 573-582 (1987).
22. Kawato, M. et al., Hierarchical neural network model for voluntary movement with application to
robotics, I EEE Control Syst ems Magazine, Vol. 8, No. 2, pp. 8-16 (April 1988).
23. Kuperstein, M., Generalized neural model for adaptive sensory-motor cont rol of single postures, I EEE
Conf. on Robotics and Aut omat i on, pp. 140-144 (1988).
24. Li ppmann, R., An i nt roduct i on to comput i ng with neural nets, I E E E A S S P Magazine, pp. 4-22 (April
1987).
25. Liu, H. et al., Building a generic architecture for robot hand control, I EEE Conf. on Neural Net works,
Vol. II, pp. 567-574 (1988).
26. Markiewicz, B., Analysis of the comput ed t orque drive met hod and comparison with conventional
position servo for a computer-controlled manipulator, Technical Memo 33-601, Jet Propulsion Lab-
oratory, Pasadena, CA (1973).
27. Marri an, C. and peckerar, M., Electronic neural net algorithm for maxi mum ent ropy deconvolution,
I EEE Conf. on Neural Net works, Vol. III, pp. 749-758 (1987).
28. Mel, B., MURPHY: a robot t hat learns by doing, Proc. A l P Neural Net works Conf., Denver,
pp. 544-553 (Nov. 1987).
29. Miller, W.T. et al., Application of a general learning al gori t hm to the cont rol of robotic manipulators,
Int. J. Robotics Research, 6, No. 2, 84-98 (1987).
30. Miyamoto, H. et al., Feedback-error-learning neural network for trajectory cont rol of a robotic
mani pul at or, Neural Net works, I, No. 3, 251-265 (1988).
31. Pao, Y. and Sobajic, D., Artificial neural-net based intelligent robotics control, Proc. SPIE-Int el l i gent
Robot s and Computer Vision, Vol. 848, pp. 542-549 (1987).
32. Pati, Y. et al., Neural networks for tactile perception, I EEE Conf. on Robotics and Automation,
pp. 134-139 (1988).
33. Psaltis, D. et al., Neural controllers, I EEE Conf. on Neural Net works, Vol. 4, pp. 551-558 (1987).
34. Psaltis, D. et al., A multitayered neural network controller, I EEE Control Syst ems Magazine, Vol. 8,
No. 2, pp. 17-21 (1988).
35. Raibert, M., Analytical equations vs table look-up for manipulation: a unifying concept, I EEE Conf.
on Decision and Control, pp. 576-579 (1977).
66 BILL HORNE ET AL.
36. Raibert, M. and Craig, J., Hybrid position/force control of manipulators, J. Dynamic Systems,
Measurement, and Control, 102, 126-133 (June 1981).
37. Rosenblatt, F., Principles of Neuro4vnamics, Spartan, New York (1962).
38. Rumelhart, D. et al., Parallel Distributed Processing: Explorations in the Microstructure of Cognition,
MIT Press, Cambridge (1986).
39. Seshadri, V., A neural network architecture for robot path planning, Proc. Second International Symp.
on Robotics and Manufacturing: Research, Foundation, and Applications, ASME Press, pp. 249-256
(1988).
40. Shepanski, J. and Macy, S., Teaching artificial neural systems to drive: manual training techniques for
autonomous systems, Proc. SPIE-Intelligent Robots and Computer Vision, Vol. 848, pp. 286-293
(1987).
41. Shepanski, J. and Macy, S., Teaching artificial neural systems to drive: manual training techniques for
autonomous systems, Proc. AI P Neural Networks Conf., Denver, pp. 693-700 (Nov. 1987).
42. Smith, F., A trainable nonlinear function generator, I EEE Trans. Automatic Control AC-11, No. 2,
212-218 (1966).
43. Sobajic, D. et al., Intelligent control of the Intelledex 605T robot manipulator, I EEE Int. Conf. on
Neural Networks, Vol. II, pp. 633-640 (1988).
44. Tank, D. and Hopfield, J., ' Neural' computation of decisions in optimization problems, Biological
Cybernetics 52, 141-152 (1985).
45. Tolat, V. and Widrow, B., An adaptive ' broom balancer' with visual inputs, I EEEI nt . Conf. on Neural
Networks, Vol. II, pp. 641-647 (1988).
46. Tsutsumi, K. and Matsumoto, H., Neural computation and learning strategy for manipulator position
control, I EEE Conf. on Neural Networks, Vol. 4, pp. 525-534 (1987).
47. Tsutsumi, K. et al., Neural computation for controlling the configuration of 2-dimensional truss
structure, I EEE Conf. on Neural Networks, Vol. II, pp. 575-586 (1988).
48. Whitney, D., Resolved motion rate control of manipulators and human prostheses, I EEE Trans.
Man-Machine Systems MMS-10, No. 2, 47-53 (1969).
49. Whitney, D., Force feedback control of manipulator fine motions, J. Dynamic Systems, Measurement,
and Control, pp. 91-97 (June 1977).
50. Widrow, B. and Smith, F., Pattern recognizing control systems, Computer and Information Sciences
(COINS) Symposium Proc., Spartan Books, Wash. DC (1963).
51. Widrow, B., The original adaptive neural net broom-balancer, I EEE Conf. on Circuits and Systems,
pp. 351-357 (1987).
52. Wilson, G. and Pawley, G., On the stability of the travelling salesman problem of Hopfield and Tank,
Biological Cybernetics 58, 63-70 (1988).
53. Yueng, D. and Bekey, G., Adaptive load balancing between mobile robots through learning in an
artificial neural system, I EEE Conf. on Decision and Control, pp. 2299-2304 (1987).