Neural Networks Lecture 9: Designing Controller Using Neural Networks

Outline Open-Loop Inverse Dynamics NN in Control Feedback Optimal Control Using Hopfield Adaptive Control Using Neural Ne
Neural Networks
Lecture 9: Designing Controller Using
Neural Networks
H.A. Talebi
Farzaneh Abdollahi
Department of Electrical Engineering

Amirkabir University of Technology
H. A. Talebi, Farzaneh Abdollahi
Neural Networks
Lecture 9
1/31
Open-Loop Inverse Dynamics

NN in Control Feedback
Gradient Through Plant
Gradient Through The Model of The Plant
Optimal Control Using Hopfield
Adaptive Control Using Neural Networks
Neural Networks
Lecture 9
2/31
Open-Loop Inverse Dynamics

I
The Inverse model obtained

from identification is directly
applied.
Considering reference
signal r
y = f 1 fr = r
This method can be

considered as Indirect
adaptive control
Neural Networks
Lecture 9
3/31
Objective: Tacking reference signal r
But: In this model, output of NN for training is not available

not be applied directly.
BP can
1
e = r y , E = e2
2
E
4wij =
wij
I
Only output of the plant, y is available.
Neural Networks
Lecture 9
4/31
Gradient Through Plant
The plant can be considered as output layer of NN with fixed weights
desired output of the NN is available and BP algorithm can be employed.

E e y
E
=
. .
wij
e y wij
E
e
= e,
= 1
e
y
y
y u
=
.
wij
u wij
To train the NN,

through plant
y
u
is required, therefore, this method is so-called Gradient

Neural Networks
Lecture 9
5/31
I
I
If the plant dynamics is not known

Solution
y
u
is not available!!
1. Using a NN identifier to identify the system dynamics directly.

I
I
y
Then apply
instead of y
.
u
u
This method is so-called Gradient Through The Model of The Plant
y
2. Approximate y
u with sign{ u } which is usually available without knowing
the dynamics
I
If the direction of the gradient is true, the magnitude of

compensated by
Neural Networks
Lecture 9
y
u
can be
6/31
An Example of Optimal Control Using Hopfield [1]

I
It can be shown that solution of the general

nonlinear programming problem with
inequality constraint: minv R n (v ) subject
to fi (v ) 0 i = 1, ..., q equivalents to
solving the canonical nonlinear
programming circuit.
It is described by the following equation

Pp
fj
i
Ci dv
j=1 g (f (v )) vi .
dt = vi
(.) and f (.) and their 1st and 2nd order

partial derivatives exist and are continuous.
g (.) is passive monotone nondecreasing fcn,

continuous, but not necessarily
differentiable, imposes the constraints in the
circuit realization.
Neural Networks
Lecture 9
7/31
Let us consider discrete time, linear quadratic regulator (LQR) with

objective:
N1
1X T
1 T
{xk Qxk + ukT Ruk }
min J = xN SxN +
uk ,xk
2
2
k=0
subject to the plant dynamics xk+1 = Axk + Buk , k = 0, ..., N 1

I
x R n, u R p
Assume (A, B) is controllable
(N+1)n+p ,
Define
v = (x0 , x1 , ..., xN , u) R
A I
0 0 . . . 0 0 B
0 ... 0
0
0 A I
0
.
.
.
0
0
0
B
.
.
.
0
0
0
0 A I . . . 0 0 0
0 ... 0
0
= .
.
.
.
.
.
..
.
.
.
.
.
..
..
.. . .
..
..
..
..
..
..
..
.
0
0
0 0 ... I
0 0
0 . . . B
0
0
0
0 0 . . . A I
0
0 . . . 0 B
The problem can be defined as minv J s.t. v = 0
Neural Networks
Lecture 9
8/31
The following theorem help us to describe equality constraint by

inequalities
P
Theorem: m equality constraints nj=1
Pnaij xj = ci for i = 1, ..., m is
equivalent
j=1 aij xj ci and
P
Pm to inequality
Pmconstraints:
n
(
a
)x
c
ij
j
i
j=1
i=1
i=1
Our problem can be presented as
P(N+1)n+p
minv J subject to f (vi ) = j=1
aij vj 0 for i = 1, ..., Nn + 1
aij are the elements of matrix A:
A
I
0 A
0
0
..
..
.
.
0
0
0
0
T
0
I
A
..
.
0
0
I
..
.
...
...
...
..
.
0
0
0
..
.
0
0
0
..
.
B
0
0
..
.
0
B
0
..
.
...
...
...
..
.
0
0
0
..
.
0 ...
I
0
0
0
. . . B
0 . . . A I
0
0
...
0
. . . 1 T T . . . T
P
Pn
Pn
T
I = (T 1), = ( n ai1 ,
ai2 , ..., i=1
i=1
i=1
P
P
Pan in ) , T
n
n
T
1 = (1, 1, . . . , 1) , = ( i=1 bi1 ,
i=1 bi2 , ...,
i=1 bin )
0
0
Neural Networks
Lecture 9
0
0
0
..
.
B
T
9/31
To implement the NN, we should find:

I
fi
vi
= aij
Pn
i = ln + 1, ..., (l + 1)n,
j=1 qiln,j vln+j
l = 0, ..., N 1
J
Pn
vi =
s
v
i
=
Nn
+ 1, ..., (N + 1)n
iNn,j
Nn+j
Pn j=1
r
v
i
=
(N
+
1)n
+ 1, ..., (N + 1)n + p
i(N+1)n,j
(N+1)n+j
j=1
qij , sij , rij are elements of Q, S, R respectively
Neural Networks
Lecture 9
10/31

1. Direct Control
I
Parameters of the controller is directly adjusted to reduce the norm of

output error
Neural Networks
Lecture 9
11/31

2. Indirect Control
I
The model of the plant is identified first and the parameters of the controller
is defined based on identified model
Neural Networks
Lecture 9
12/31
They can be controller y Neural networks
Neural Networks
Lecture 9
13/31
Example [2]
Consider the difference equation:

yp (k + 1) = f [yp (k), yp (k 1)] + u(k)
f (.) is unknown
For the sake of simulation f [yp (k), yp (k 1)] =
Reference model: ym (k + 1) = 0.6ym (k) + 0.2ym (k 1) + r (k)
r (k) = sin( 2k
25 ): a bounded reference input
Objective: Determine a bound control signal u(k) s.t.

limk ec (k) = yp (k) ym (k) = 0
Neural Networks
Lecture 9
yp (k)yp (k1)[yp (k)+2.5]

1+yp2 (k)+yp2 (k1)
14/31
Example Contd
If f (.) was known the proper control signal would be

u(k) = f [yp (k), yp (k 1)] + 0.6yp (k) + 0.2yp (k 1) + r (k) yields
ec (k + 1) = 0.6ec (k) + 0.2ec (k 1)
I
the reference model is a.s. since limk ec (k) = 0
Since the plant is unknown, assuming the unforced system is stable, f (.)
is estimated by series parallel NN identifier as f(.)
Hence u(k) = f[yp (k), yp (k 1)] + 0.6yp (k) + 0.2yp (k 1) + r (k)
Neural Networks
Lecture 9
15/31
Example Contd
I
Identification will be off-line
Once the plant is identified in desired level of accuracy, control is initiated

to make the plant output follow the reference model.
Note that using the estimated function in fb loop may result in

unbounded solution
Hence for on-line control, identification and control should proceed

simultaneously.
The time interval Ti and Tc for updating the identification and control
parameters should be chosen wisely.
Neural Networks
Lecture 9
16/31
Example Contd
I
a) Identified signal yp (dashed) and output of the plant with no control

action (solid)
b) Response for r = sin( 2k

25 ) with control (dashed); reference signal
(solid)
Ti = Tc = 1
Neural Networks
Lecture 9
17/31
Example Contd
I
Choose Ti = Tc = 10
Response for r = sin( 2k

25 ) with control (dashed); reference signal (solid)
To have stable on-line control, the identification should be accurate

enough before the control action is initiated!
Neural Networks
Lecture 9
18/31
Example 2 [2]
I
Consider the difference equation:

Pm1
yp (k + 1) = f [yp (k), yp (k 1), ..., yp (k n + 1)] + j=0
j u(k j) m n
f (.) and j are unknown; 0 is nonzero with known sign
For the sake of simulation

f [yp (k), yp (k 1)..., yp (k n + 1)] =
0 = 1, 1 = 0.8
5yp (k)yp (k1)

;
1+yp2 (k)+yp2 (k1)+yp2 (k2)
Reference model:
ym (k + 1) = 0.32ym (k) + 0.64ym (k 1) 0.5ym (k 2) + r (k)
r (k) = sin( 2k
25 ): a bounded reference input
Objective: Determine a bound control signal u(k) s.t.

limk ec (k) = yp (k) ym (k) = 0
Assume sgn(0 ) = +1; 0 0.1
Neural Networks
Lecture 9
19/31
Example 2 Contd
I
The control signal is: u(k) = 1 [fk [yp (k), yp (k 1), yp (k 2)]
0
1 u(k 1) + 0.32yp (k) + 0.64yp (k 1) 0.5yp (k 2) + r (k)]
Choose Ti = Tc = 10
Response for r = sin( 2k

25 ) with control (dashed); reference signal (solid)
left): first 100 sec; right) after 9900sec
Neural Networks
Lecture 9
20/31
Example 3 [2]
Consider dynamics similar to Example 2 but replace 0.8u(k 1) with

1.1u(k 1)
Apply similar controller
The system is nonminimum phase (it has zero out of unit circle)
The output error is bounded but the control signal is unbounded
Neural Networks
Lecture 9
21/31
Example 3 Contd
I
left) Response for r = sin( 2k

25 ) with control (dashed); reference signal
(solid)
right) control signal u(k)
Neural Networks
Lecture 9
22/31
Inverse Dynamics Model Learning (IDML) [3]
I
I
Consider the system dynamics x = f (x, x)

+ u, y = x
To track reference signal ya , control signal can be defined u = un + uc
I
I
uc = (
ya ) + K1 (y r y a ) + K0 (yr ya ) is conventional FB controller
un = f (x, x)
f (.) is not known and is estimated by NN
Neural Networks
un = f
Lecture 9
23/31
The error signal for training is en = u un = uc
f
The learning rule is w = w
uc
Neural Networks
Lecture 9
24/31
Example 4 [3]
Consider
one-link
flexible arm:

, )
+ F1 + fc
h1 (,
u
M() +
=
h2 (, ) + K + F2
I
I
I
I
I
I
I
: hub angle
: deflection variable
h1 and h2 are Coriolis and Centrifugal forces, respectively
M(): P.D. inertia matrix
u: torque
F1 : viscus damping; F2 damping matrix;
fc hub friction; K stiffness matrix
Neural Networks
Lecture 9
25/31
Example 4 Contd
I
I
The nonlinear dynamics is assumed to be unknown

For the sake of simulation, the numerical values
are
I
I
I
m()
1.0703 0.0282
1.6235 0.4241 ;
M() = 1.0703
0.0282 0.4241
2.592
2
2
m() = 0.9929 + 0.12(1 + 2 ) 0.241 2
17.4561
0
K=
0
685.5706
, )
= 0.24[(
1 2 )1 (1 2 )2 ]
h1 (,

0.122 (1 2 )
h2 (, ) =
0.122 (2 1 )

4.74 > 0
fc = Ccoul ( 1+e210 1); Ccoul =
4.77 < 0
By output redefinition, the nonminimum phase problem is solved
Neural Networks
Lecture 9
26/31
Example 4 Contd
The NN structure:
I
Three layer: 4 input; 5 hidden,1 output
K0 = 1; K1 = 2
Neural Networks
Lecture 9
27/31
Example 4 Contd
Neural Networks
Lecture 9
28/31
Example 4 Contd
Neural Networks
Lecture 9
29/31
Example 4 Contd
Neural Networks
Lecture 9
30/31
References
M P. Kennedy and L. O. Chua, Neural networks for nonlinear programming, IEEE

Transactions of Circuit and Systems, vol. 35, no. 5, pp. 554562, 1988.
K.S. Narendra and K. Parthasarathy, Identification and control of dynamical systems
using neural networks, IEEE Trans. on Neural Networks, vol. 1, no. 1, pp. 427, March
1990.
H.A. Talebi, .R.V Patel and K. Khorasani, Control of Flexible-link Manipulators Using
Neural Networks.
Springer, 2001.
Neural Networks
Lecture 9
31/31

Neural Networks Lecture 9: Designing Controller Using Neural Networks

Transféré par

Informations du document

Description originale:

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Neural Networks Lecture 9: Designing Controller Using Neural Networks

Transféré par

Droits d'auteur :

Formats disponibles

Outline Open-Loop Inverse Dynamics NN in Control Feedback Optimal Control Using Hopfield Adaptive Control Using Neural Ne

Department of Electrical Engineering

H. A. Talebi, Farzaneh Abdollahi

Open-Loop Inverse Dynamics

H. A. Talebi, Farzaneh Abdollahi

Open-Loop Inverse Dynamics

The Inverse model obtained

This method can be

H. A. Talebi, Farzaneh Abdollahi

Objective: Tacking reference signal r

But: In this model, output of NN for training is not available

Only output of the plant, y is available.

H. A. Talebi, Farzaneh Abdollahi

Gradient Through Plant

The plant can be considered as output layer of NN with fixed weights

desired output of the NN is available and BP algorithm can be employed.

To train the NN,

H. A. Talebi, Farzaneh Abdollahi

is required, therefore, this method is so-called Gradient

If the plant dynamics is not known

1. Using a NN identifier to identify the system dynamics directly.

If the direction of the gradient is true, the magnitude of

H. A. Talebi, Farzaneh Abdollahi

An Example of Optimal Control Using Hopfield [1]

It can be shown that solution of the general

It is described by the following equation

(.) and f (.) and their 1st and 2nd order

g (.) is passive monotone nondecreasing fcn,

H. A. Talebi, Farzaneh Abdollahi

Let us consider discrete time, linear quadratic regulator (LQR) with

subject to the plant dynamics xk+1 = Axk + Buk , k = 0, ..., N 1

Assume (A, B) is controllable

H. A. Talebi, Farzaneh Abdollahi

The following theorem help us to describe equality constraint by

aij are the elements of matrix A:

H. A. Talebi, Farzaneh Abdollahi

To implement the NN, we should find:

j=1 qiln,j vln+j

H. A. Talebi, Farzaneh Abdollahi

Adaptive Control Using Neural Networks

Parameters of the controller is directly adjusted to reduce the norm of

H. A. Talebi, Farzaneh Abdollahi

Adaptive Control Using Neural Networks

H. A. Talebi, Farzaneh Abdollahi

They can be controller y Neural networks

H. A. Talebi, Farzaneh Abdollahi

Consider the difference equation:

For the sake of simulation f [yp (k), yp (k 1)] =

Reference model: ym (k + 1) = 0.6ym (k) + 0.2ym (k 1) + r (k)

Objective: Determine a bound control signal u(k) s.t.

H. A. Talebi, Farzaneh Abdollahi

yp (k)yp (k1)[yp (k)+2.5]

If f (.) was known the proper control signal would be

the reference model is a.s. since limk ec (k) = 0

H. A. Talebi, Farzaneh Abdollahi

Identification will be off-line

Once the plant is identified in desired level of accuracy, control is initiated

Note that using the estimated function in fb loop may result in

Hence for on-line control, identification and control should proceed

H. A. Talebi, Farzaneh Abdollahi

a) Identified signal yp (dashed) and output of the plant with no control

b) Response for r = sin( 2k