Vous êtes sur la page 1sur 43

I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

I NTRODUCTION TO O PTIMAL C ONTROL


S YSTEMS

Harry G. Kwatny

Department of Mechanical Engineering & Mechanics


Drexel University

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

O UTLINE
I NTRODUCTION
Preliminaries
Bibliography
T HE O PTIMAL C ONTROL P ROBLEM
Definition
Examples
O PTIMIZATION BASICS
Local Extrema
Constraints
VARIATIONAL C ALCULUS
Classical Variational Calculus
Free Terminal Time
Systems with Constraints

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

P RELIMINARIES

W HAT IS O PTIMAL C ONTROL ?

I Optimal control is an approach to control systems design that


seeks the best possible control with respect to a performance
metric.
I The theory of optimal control began to develop in the WW II
years. The main result of this period was the Wiener-Kolmogorov
theory that addresses linear SISO systems with Gaussian noise.
I A more general theory began to emerge in the 1950s and 60s
I In 1957 Bellman published his book on Dynamic Programming
I In 1960 Kalman published his multivariable generalization of
Wiener-Kolmogorov
I In 1962 Pontryagin et al published the maximal principle
I In 1965 Isaacs published his book on differential games

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

P RELIMINARIES

C OURSE C ONTENT
1. Introduction
I Optimization basics
I Intro to Variational Calculus
2. Variational Calculus and the Minimum Principle
I Unconstrained Control problems
I Control and State Constraints
3. Dynamic programming
I Principle of Optimality
I The Hamilton-Jacobi-Bellman Equation
4. Min-Max Optimal Control
I Min-Max Control
I Game Theory
5. Hybrid Systems
I Hybrid Systems Basics
I Hybrid Systems Optimal Control

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

B IBLIOGRAPHY

B IBLIOGRAPHY

I E. B. Lee and L. Markus, Foundations of Optimal Control


Theory. New York: John Wiley and Sons, Inc., 1967.
I A. E. Bryson and Y.-C. Ho, Applied Optimal Control.
Waltham: Blaisdell, 1969.
I S. J. Citron, Elements of Optimal Control. New York: Holt,
Rinehart and Winston, Inc., 1969.
I D. E. Kirk, Optimal Control Theory: An Introduction.
Englewood Cliffs, NJ: Prentice-Hall, 1970. (now available
from Dover)

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

D EFINITION

P ROBLEM D EFINITION
We will define the basic optimal control problem:
I Given system dynamics

x = f (x, u) , x X Rn , u U Rm

I Find a control u (t) , t [0, T] that steers the system from


an initial state x (0) = x0 to a target set G and minimizes the
cost Z T
J (u ()) = gT (x (T)) + g (x (t) , u (t)) dt
0

R EMARK
gT is called the terminal cost and g is the running cost. The terminal
time T can be fixed or free. The target set can be fixed or moving.

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

D EFINITION

O PEN L OOP VS . C LOSED L OOP


I If we are concerned with a single specified initial state x0 , then
we might seek the optimal control control u (t), u : R Rm that
steers the system from the initial state to the target. This is an
open loop control.
I On the other hand, we might seek the optimal control as a
function of the state u (x), u : Rn Rm . This is a closed loop
control; sometimes called a synthesis.
I The open loop control is sometimes easier to compute, and the
computations are sometimes performed online a method
known as model predictive control.
I The closed loop control has the important advantage that it is
robust with respect to model uncertainty, and that once the
(sometimes difficult) computations are performed off-line, the
control is easily implemented online.

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

E XAMPLES

E XAMPLE A M INIMUM T IME P ROBLEM


Consider steering a unit mass, with bounded applied control
force, from an arbitrary initial position and velocity to rest at the
origin in minimum time. Specifically,

x = v
v = u, |u| 1

The cost function is Z T


J= dt T
0

R EMARK
This is an example of a problem with a control constraint.

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

E XAMPLES

E XAMPLE A M INIMUM F UEL P ROBLEM


Consider the decent of a moon lander.
h = v
u
v = g + m
m = ku
The thrust u is used to steer the system to h = 0, v = 0. In
addition we wish to minimize the fuel used during landing, i.e.
Z t
J= k udt
0
Furthermore, u is constrained, 0 u c, and the state
constraint h 0 must be respected.
R EMARK
This problem has both control and state constraints.

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

E XAMPLES

E XAMPLE A L INEAR R EGULATOR P ROBLEM

Consider a system with linear dynamics

x = Ax + Bu

We seek a feedback control that steers the system from an


arbitrary initial state x0 towards the origin in such a way as to
minimize the cost
Z T
T 1  T
x (t) Qx (t) + uT (t) Ru (t) dt

J = x (T) QT x (T) +
2T 0

The final time T is considered fixed.

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

E XAMPLES

E XAMPLE A ROBUST S ERVO P ROBLEM


Consider a system with dynamics
x = Ax + B1 w + B2 u
z = C1 x + D11 w + D12 u
y = C2 x + D21 w + D22 u
where w is an external disturbance. The goal is to find an output (y)
feedback synthesis such that the performance variables (process
errors) z remain close to zero. Note that w (t) can be characterized in
several ways, stochastic (the H2 problem)
Z 
T
J=E z (t) z (t) dt

or deterministic (the H problem)
Z
J = max zT (t)z(t)dt
kwk2 =1

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

L OCAL E XTREMA

D EFINITIONS

Consider the scalar function

f (x) , x Rn

which is defined and smooth on a domain D Rn . We further


assume that the region D is defined by a scalar inequality
(x) 0, i.e.,

intD = {x Rn | (x) < 0 }


D = {x Rn | (x) = 0 }

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

L OCAL E XTREMA

D EFINITIONS
D EFINITION (L OCAL M INIMA & M AXIMA )
An interior point x intD is a local minimum if there exists a
neighborhood U of x such that

f (x) f (x ) x U

It is a local maximum if

f (x) f (x ) x U

Similarly, for a point x D, we use a neighborhood U of x


within D. With this modification boundary local minima and
maxima are defined as above.

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

L OCAL E XTREMA

O PTIMAL I NTERIOR P OINTS


I Necessary conditions. A point x intD is an extremal
point (minimum or maximum) only if

f
(x ) = 0
x
I Sufficient conditions. x is a minimum if
2f
(x ) > 0
x2
a maximum if
2f
(x ) < 0
x2

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

C ONSTRAINTS

O PTIMIZATION WITH CONSTRAINTS N ECESSARY


C ONDITIONS
We need to find extremal points of f (x), with x D. i.e., find
extremal points of f (x) subject to the constraint (x) = 0.
Consider a more general problem where there are m constraints, i.e.,
: Rn Rm .
Let be an m-dimensional constant vector (called Lagrange
multipliers) and define the function

H (x, ) = f (x) + T (x)
Then x is an extremal point only if
H (x , ) H (x , )
= 0, (x) = 0
x
Note there are n + m equations in n + m unknowns x,

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

C ONSTRAINTS

S IGNIFICANCE OF THE L AGRANGE M ULTIPLIER


Consider extremal points of f (x1 , x2 ) subject to the single constraint
(x1 , x2 ) = 0. At an extremal point (x1 , x2 ) we must have
f (x1 , x2 ) f (x1 , x2 )
df (x1 , x2 ) = dx1 + dx2 = 0 (1)
x1 x2
but dx1 and dx2 are not independent. They satisfy
(x1 , x2 ) (x1 , x2 )
d (x1 , x2 ) = dx1 + dx2 = 0 (2)
x1 x2
From (1) and (2) it must be that
f (x1 , x2 )/x1 f (x1 , x2 )/x2

= =
(x1 , x2 )/x1 (x1 , x2 )/x2
Accordingly, (1) and (2) yield
f (x1 , x2 ) (x1 , x2 ) f (x1 , x2 ) (x1 , x2 )
+ = 0, + =0
x1 x1 x2 x2

R EMARK
Note that these are the form of the previous slide.

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

C ONSTRAINTS

O PTIMIZATION WITH C ONSTRAINTS S UFFICIENT


C ONDITIONS
H 1 2H
df = dx + dxT 2 dx T d + h.o.t.
x 2 x
but

d = dx = 0 dx ker dx = d
x x
(x)
= span ker
x
Thus, for x extremal
1 T T 2 H (x )
df (x ) = d d + h.o.t.
2 x2
2 H (x )
T > 0 min
x2
2
H (x )
T < 0 max
x2
O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

C ONSTRAINTS

E XAMPLE
 
f (x1 , x2 ) = x1 x12 + 2x22 1 , (x1 , x2 ) = x12 + x22 1

Interior:
(x1 , x2 , f ) = (0, 0.707, 0) (0, 0.707, 0) (0.577, 0, 0.385) (0.577, 0. 0.385)
Boundary:

(x1 , x2 , , f ) = (0.577, 0.8165, 1.155, 0.385) (0.577, 0.8165, 1.155, 0.385)


(0.577, 0.8165, 1.155, 0.385) (0.577, 0.8165, 1.155, 0.385)
(1, 0, 1, 0) (1, 0, 1, 0)

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

C LASSICAL VARIATIONAL C ALCULUS

O PTIMIZING A T IME T RAJECTORY

I We are interested in steering a controllable system along a


trajectory that is optimal in some sense.
I Three methods are commonly used to address such
problems:
I The calculus of variations
I The Pontryagin maximal Principle
I The principle of optimality and dynamic programming
I The calculus of variations was first invented to characterize
the dynamical behavior of physical systems governed by a
conservation law.

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

C LASSICAL VARIATIONAL C ALCULUS

C ALCULUS OF VARIATIONS : L AGRANGIAN S YSTEMS


A Lagrangian System is characterized as follows:
I The system is define in terms of a vector of configuration
coordinates, q, associated with velocities q.
I The system has kinetic energy T (q, q) = qT M (q) q/2, and
potential energy V (q) from which we define the Lagrangian

L (q, q) = T (q, q) V (q)

I The system moves along a trajectory q (t), between initial


and final times t1 , t2 in such a way as to minimize the
integral Z t2
J (q (t)) = L (q (t) , q (t)) dt
t1

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

C LASSICAL VARIATIONAL C ALCULUS

E XAMPLES OF L AGRANGIAN S YSTEMS

T = 12 qT M (q) q
 2 M (q) = 
`1 (m1 + m2 ) + `22 m2 + 2`1 `2 m2 cos 2 `2 m2 (`2 + `1 cos 2 )
`2 m2 (`2 + `1 cos 2 ) `22 m2
V (q) = m1 g (g`1 (m1 + m2 ) sin 1 + g`2 m2 sin (1 + 2 ))

1 m1

2
m2

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

C LASSICAL VARIATIONAL C ALCULUS

N ECESSARY C ONDITIONS : F IXED T ERMINAL T IME 1


I A real-valued, continuously differentiable function q (t) on the
interval [t1 , t2 ] will be called admissible.
I Let q (t) be an optimal admissible trajectory and q (t, ) a not
necessarily optimal trajectory, with
q (t, ) = q (t) + (t)
where > 0 is a small parameter and (t) is arbitrary.
I Then
Z t2
J (q (t, )) = L (q (t) + (t) , q (t) + (t)) dt
t1

I An extremal of J is obtained from



J (q (t, ))
J (q (t)) = =0

=0

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

C LASSICAL VARIATIONAL C ALCULUS

F IXED T ERMINAL T IME 2

t2
L (q (t) , q (t)) L (q (t) , q (t))
Z  
J (q (t)) = (t) + (t) dt
t1 q q

We can apply the integration by parts formula


Z Z
udv = uv vdu

to the first term to obtain



Rt  L(q (t),q (t))

J (q (t)) = t12 dtd L(q (t),q
q
(t))
+ (t) dt
t2q
L(q (t),q (t))
+ (t)

q
t1

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

C LASSICAL VARIATIONAL C ALCULUS

F IXED T ERMINAL T IME 3


Now, set J (q (t)) = 0 to obtain:
I the Euler-Lagrange equations

d L (q (t) , q (t)) L (q (t) , q (t))


=0
dt q q
I the transversality conditions
L (q (t1 ) , q (t1 )) L (q (t2 ) , q (t2 ))
q (t1 ) = 0, q (t2 ) = 0
q q

R EMARK
These results allow us to treat problems in which the initial and terminal times are fixed
and individual components of q(t1 ) and q(t2 ) are fixed or free. Other cases of interest
include: 1) the terminal time is free, and 2) the terminal time is related to the terminal
configuration, e.g., by a relation (q (t2 ) , t2 ) = 0.

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

N ECESSARY C ONDITIONS : F REE T ERMINAL T IME 1


Consider the case of fixed initial state and free terminal time. Let q (t) be the
optimal trajectory with optimal terminal time t2 . The perturbed trajectory
terminates at time t2 + t2 . Its end state is q (t2 ) + (t2 + ). The
perturbed cost is
Z t +t
2
J (q , q, t) = L (q (t) + q (t) , q (t) + q (t)) dt
t1

From this we obtain:


 
R t L(q (t),q (t)) L(q (t),q (t)) L(q (t1 ),q (t1 ))
J = t 2 dtd q
+ q
q (t) dt q
q (t1 )
1
L(q (t2 ),q (t2 ))
+ q
q (t2 ) + L (q (t2 + t) , q (t2 )) t

Now, we want to allow both the final time and the end point to vary. The
actual end state is:

q2 = q (t2 + t) = q (t2 ) + q (t2 ) t

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

F REE T ERMINAL T IME 2


 
R t L(q (t),q (t)) L(q (t),q (t)) L(q (t1 ),q (t1 ))
J = t1
2
dtd q
+ q
q (t) dt q
q (t1 )
 
L(q (t2 ),q (t2 )) L(q (t2 ),q (t2 ))
+ q
q (t2 ) + L (q (t2 ) , q (t2 )) q
q (t2 ) t

Thus, we have have the Euler-Lagrange Equations, as before, but the


transversality conditions become:
I the previous conditions:

L (q (t1 ) , q (t1 )) L (q (t2 ) , q (t2 ))


q (t1 ) = 0, q (t2 ) = 0
q q
I plus, additional conditions:

L (q (t2 ) , q (t2 ))
 
L (q (t2 ) , q (t2 )) q (t2 ) t = 0
q

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

F REE T ERMINAL T IME 3


Ordinarily, the initial state and time are fixed so that the transversality
conditions require
q (t1 ) = 0
For free terminal time t is arbitrary. Thus from the transversality conditions:
I If the terminal state is fixed, q (t2 ) = 0 the terminal condition is:

L (q (t2 ) , q (t2 ))
L (q (t2 ) , q (t2 )) q (t2 ) = 0
q

I If the terminal state is free, q2 arbitrary the terminal condition is:

L (q (t2 ) , q (t2 ))
= 0, L (q (t2 ) , q (t2 )) = 0
q

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

C ONTROL E XAMPLE
First note that in the elementary variational calculus, we could
simply replace q by x, and add the definition x = u, so that the
cost function becomes
Z t2
J (x (t)) = L (u (t) , x (t)) dt
t1

Hence, we have a simple control problem. We consider 3


variants:
1. Free endpoint, fixed terminal time
2. Fixed endpoint, fixed terminal time
3. Fixed endpoint, free terminal time

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

E XAMPLE 1: F REE ENDPOINT, F IXED TERMINAL TIME


Suppose x R, t1 = 0, t2 = 1, and L = x2 + u2 /2:


Z 1
1 2
x + u2 dt

J (x (t)) =
0 2

The Euler-Lagrange equations become:


 
d L L
= 0 u x = 0
dt u x

The transversality conditions are:

L (x (1) , u (1))
x (0) = x0 , =0
u

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

E XAMPLE 1, C ONT D
Thus, we have the equations
    
x 0 1 x
=
u 1 0 u
L (x (1) , u (1))
x (0) = x0 , = 0 u (1) = 0
u
Consequently,

 
0 1 t      
x (t) 1 0 a x (t) a cosh (t) + b sinh (t)
=e =
u (t) b u (t) b cosh (t) + a sinh (t)

x0 e2 x0
a = x0 , b =
1 + e2

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

E XAMPLE 1: F IXED ENDPOINT, F IXED TERMINAL TIME

Suppose we consider a fixed end point, say x (t2 ) = 0 with the


terminal time still fixed at t2 = 1. Then the Euler-Lagrange
equations remain the same, but the boundary conditions
change to:
x (0) = x0 , x (1) = 0
Thus, we compute

x0 + e2 x0
a = x0 , b =
1 + e2

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

E XAMPLE 1: F IXED ENDPOINT, F REE TERMINAL TIME

Suppose we consider a fixed end point, say x (t2 ) = xf with the


terminal time t2 free. Then the Euler-Lagrange equations
remain the same, but the boundary conditions change to:

x (0) = x0 , x (t2 ) = xf , (L Lu u)|t2 = 0

From this we compute

a = x0 , a2 = b2 , a cosh (t2 ) + b sinh (t2 ) = xf

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

E XAMPLE 1, C ONT D
The figures below show the optimal control and state
trajectories from the initial state x0 = 1,xf = 0, with t2 = 1 for the
fixed time case. For the free time case xf = 0.01.
Free terminal state, Fixed Fixed terminal state, Fixed terminal state,
time: Fixed Time: Free time:
x,u x,u

x,u 1.0 1.0

1.0
0.5
0.5

0.5
t
0.2 0.4 0.6 0.8 1.0 t
1 2 3 4 5
-0.5
t
0.2 0.4 0.6 0.8 1.0 -0.5

-1.0
-0.5 -1.0

R EMARK
In the free time case, with xf = 0.01, the final time is t2 = 4.60517. With xf = 0 the final
time is t2 = . Note that the case of free terminal state and free terminal time (not
shown) is trivial with t2 = 0.

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

VARIATIONAL C ALCULUS WITH D IFFERENTIAL


C ONSTRAINTS
Systems with nonintegrable differential , i.e., nonholonomic,
constraints have been treated by variational methods. As
before, we seek extremals of the functional:
Z t2
J (q (t)) = L (q (t) , q (t) , t) dt
t1

But now subject to the constraints:

(q (t) , q (t) , t) = 0

where : Rn Rn R Rm .

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

D IFFERENTIAL C ONSTRAINTS , C ONT D


The constraints are enforce for all time, so introduce the m
Lagrange multipliers (t) and consider the modified functional
Z t2
J (q (t)) = L (q (t) , q (t) , t) + T (t) (q (t) , q (t) , t) dt
t1

We allow variations in both q and and t2 , Taking the variation


and integrating by parts yields
Rt 
J = t12 dtd Lq + T q + Lq + T q q (t) + T dt
    

+ Lq + T q t q2 + L + T q Lq + T q q t t2
      
2 2

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

N ECESSARY C ONDITIONS WITH C ONSTRAINTS


Again we have the Euler-Lagrange equations, now in the form:
d 
Lq + T q Lq + T q = 0
  
dt
The differential constraints:

(q, q, t) = 0

and the boundary conditions

Lq + T q t q2 = 0
 
2

With free terminal time, we also have

L + T q Lq + T q q t = 0
    
2

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

E XAMPLE
Once again consider the problem
Z t2 1 2 
x = u, J (x (t)) = x + u2 dt
0 2

But now, add the constraint

(x, x) = x2 1 u2 1 = 0 u = 1

The Euler-Lagrange equations now become

d  x
Lu + T u Lx + T x = 0 =
  
dt 2u

R EMARK
I Note that u = 1 u = 0 almost everywhere,
I Also, t u2 dt = t
R
0

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

E XAMPLE C ONTINUED

We now consider three cases:


I free end point, fixed terminal time
I fixed end point, fixed terminal time
I fixed end point, free terminal time

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

E XAMPLE : F REE E ND P OINT, F IXED TERMINAL T IME

Steer from x (0) = x0 .

I system x = u
x
I Euler equation = 2u
I constraint u = 1
I initial condition x (0) = x0
= 0 (1) = 21
 
I terminal condition Lu + T u
t2

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

E XAMPLE : F REE E ND P OINT, F IXED TERMINAL T IME 2


The control is piecewise constant, switching between u = +1 and
u = 1. Assume:
I there is only one switch that takes place at t = T with 0 < T < 1 with
u(0) = u0 = 1 and u(t) = u0 , t > T
I (0) = 0

x0 + u0 t 0<t<T
x(t) =
x0 + u0 T u0 (t T) T < t 1
1
x0 + u20t2
( 
0 + 2u 0
0<t<T
(t) = 
2
0 + 2u10 x0 + u20 T 2 2u x0
+ 12 T (t T) + 14 (t T)

0
T<t1
 
1 3 x0
(1) = 0 + + 1 T + T2 = 0
2 4 2u0

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

E XAMPLE : F REE E ND P OINT, F IXED TERMINAL T IME 3


Consider the case x(0) = 0. The necessary conditions are satisfied with
   
3 1 1 1p
(u0 = 1) 0 T= 2 40
4 2 2 2

Two examples are given below. Only the case 0 = 21 , T = 1


2
is optimal.

1 3
0 = 0 =
2 4

: :
0.5
0.5

0.0
0.0

x,,J
x,,J

-0.5
-0.5

-1.0
-1.0 0.0 0.2 0.4 0.6 0.8 1.0
0.0 0.2 0.4 0.6 0.8 1.0
t
t

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

E XAMPLE : F IXED E ND P OINT, F IXED TERMINAL T IME

Steer from x (0) = x0 to x (1) = x1 .

I system x = u
I Euler equation = x
2u
I constraint u = 1
I boundary conditions x (0) = x0 , x (1) = x1

Assume single switch at time T

x0 + kT k(1 T) = x1 (k = 1) 0 T 1
1 + x1 x0 1 + x1 (k = 1) T = kx2k0 +x1

O PTIMAL C ONTROL
I NTRODUCTION T HE O PTIMAL C ONTROL P ROBLEM O PTIMIZATION BASICS VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

E XAMPLE : F REE TERMINAL T IME , F IXED E ND P OINT


Steer from x (0) = x0 to x (t2 ) = 0
I system x = u
I Euler equation = x
2u
I constraint u = 1
I initial condition x (0) = x0 , x (t2 ) = 0
I terminal time condition L + T x Lu T u = 0 (t2 ) = 1 1
 
4k
+ 2
!
  2
t2 kx0 (t2 2T) 1
(x0 0, t2 x0 ) (k = 1) T = 0 =
2 4

!
 2
t2 kx0 (t2 2T) 1
(x0 0, t2 x0 )(k = 1) T = 0 =
2 4

O PTIMAL C ONTROL