Optimal Control PDF

Section 3 : Introduction to Optimal Control
November 21, 2017

Objective of optimal control theory
To determine the control signals that will cause a

system/process to satisfy the physical constraint and at
the same time minimize (or maximize) some
performance criterion.
A control system can be any time-dependent process:

it can be as simple as a system of travelling from one place to
another which minimizes total cost; or
as complex as the design of a spacecraft attitude control system
that minimizes fuel consumption.
AMAT 112 Introduction to Optimization Modelling







Optimal Control Problem Formulation
1 A mathematical description (or model) of the process

to be controlled;
2 A statement of the physical constraint; and
3 Specication of a performance criterion (objective).


to be controlled;


to be controlled;

The Mathematical Model
Suppose we have a process dened by the following:

State variables (or simply states)
x1 (t), x2 (t), . . . , xn (t)
of the process at time t; and
Control inputs
u1 (t), u2 (t), . . . , un (t)

to the process at time t.
NOTE: xi and ui are functions of time t

Suppose we have a process dened by the following:

State variables (or simply states)
x1 (t), x2 (t), . . . , xn (t)
of the process at time t; and
Control inputs
u1 (t), u2 (t), . . . , un (t)

to the process at time t.
NOTE: xi and ui are functions of time t

Given the states of the system and control inputs, the system can be
describe by n rst order dierential equations:
ẋ1 (t) = a1 (x1 (t), x2 (t), . . . , xn (t), t)

ẋ2 (t) = a2 (x1 (t), x2 (t), . . . , xn (t), t)
....
..
ẋn (t) = an (x1 (t), x2 (t), . . . , xn (t), t) (1)
NOTE: ẋi (t) is just another notation for the derivative of xi with
respect to time t (Newton's notation of dierentiation)

To simplify, using vector notations, we shall dene:
the state vector of the system
 
x1 (t)
 x2 (t) 
x(t) =  . 
.
 
 . 
xn (t)
; and
the control vector given by
 
u1 (t)
 u2 (t) 
u(t) =  . 
 .. 
 
un (t)
Now, the state equations can be written as
ẋ(t) = a(x(t), u(t), t).

 
x1 (t)
 x2 (t) 
x(t) =  . 
.
 
 . 
xn (t)
; and
 
u1 (t)
 u2 (t) 
u(t) =  . 
 .. 
 
un (t)
ẋ(t) = a(x(t), u(t), t).

 
x1 (t)
 x2 (t) 
x(t) =  . 
.
 
 . 
xn (t)
; and
 
u1 (t)
 u2 (t) 
u(t) =  . 
 .. 
 
un (t)
ẋ(t) = a(x(t), u(t), t).

Example (Example of a control system)
Suppose we have a parked car to be driven in a straight line away from

point O. The distance of the car from point O at time t is denoted by
d(t).
To simplify the model, assume that the car is a point with negligible
mass on the number line that can be accelerated by using the throttle
or decelerated using the break.

The resulting dierential equation that describes the change in

distance will be
˙ = α(t) + β(t)
d(t)
where the controls α is throttle acceleration and β is breaking
deceleration.
Selecting position and velocity as state variables, that is

x1 = d(t) and x2 = d(t)
˙
and letting
u1 (t) = α(t) and u2 (t) = β(t),

distance will be
˙ = α(t) + β(t)
d(t)
deceleration.

x1 = d(t) and x2 = d(t)
˙
and letting
u1 (t) = α(t) and u2 (t) = β(t),

distance will be
˙ = α(t) + β(t)
d(t)
deceleration.

x1 = d(t) and x2 = d(t)
˙
and letting
u1 (t) = α(t) and u2 (t) = β(t),
We nd the state equations become

ẋ1 (t) = x2 (t)
ẋ2 (t) = u1 (t) + u2 (t),
or using matrix notation, we have

0 1 0 0
ẋ(t) = x(t) + u(t).
0 0 1 1
Mathematical model of the process in state form


ẋ1 (t) = x2 (t)
ẋ2 (t) = u1 (t) + u2 (t),

0 1 0 0
ẋ(t) = x(t) + u(t).
0 0 1 1


ẋ1 (t) = x2 (t)
ẋ2 (t) = u1 (t) + u2 (t),

0 1 0 0
ẋ(t) = x(t) + u(t).
0 0 1 1

Suppose t ∈ [t0 , tf ], we have the following denition:

Denition (1. Control)
A history of control input values during the interval [t0 , tf ] denoted by
u is called a control history, or simply a control.
Denition (2. State Trajectory)

A history of state values during the interval [t0 , tf ] is called a state
trajectory and is denoted by x.
The terms "history", "curve", "function" and "trajectory" will be

used interchangeably.
It is more important to keep mind the dierence between a
function and the value of a function.














Example (Function and Function value)
The gure above shows a single-valued function of time which

is denote by x.
The value of the function at time t1 is denoted by x(t1 ).



is denote by x.



is denote by x.



is denote by x.

Physical Constraints
After we have a general mathematical model, the next step in optimal

control is to dene the physical constraints on the state and control
values.
Example (Car driving problem)
Consider our previous example of driving a car between points O and e.

Assume that the car starts from rest and stops upon reaching point e.

First, let us dene the state constraints .

If t0 is the time of leaving O, and tf is the time of arrival at point e,
then
x1 (t0 ) = 0
x1 (tf ) = e.
In addition, since the car starts from rest and stops at e
x2 (t0 ) = 0
x2 (tf ) = 0.


then
x1 (t0 ) = 0
x1 (tf ) = e.
x2 (t0 ) = 0
x2 (tf ) = 0.


then
x1 (t0 ) = 0
x1 (tf ) = e.
x2 (t0 ) = 0
x2 (tf ) = 0.


then
x1 (t0 ) = 0
x1 (tf ) = e.
x2 (t0 ) = 0
x2 (tf ) = 0.

In matrix notation, these boundary conditions are

0 e
x(t0 ) =
0
=0 and x(tf ) =
0
Assuming that the car does not back-up, we have additional

constraints
0 ≤x1 (t) ≤ e
0 ≤x2 (t)


0 e
x(t0 ) =
0
=0 and x(tf ) =
0

constraints
0 ≤x1 (t) ≤ e
0 ≤x2 (t)


0 e
x(t0 ) =
0
=0 and x(tf ) =
0

constraints
0 ≤x1 (t) ≤ e
0 ≤x2 (t)

Next are the constraints on the control inputs (acceleration)?

We know that all cars have acceleration limits which depends on
the engine capability.
Also, deceleration is is limited by the braking system of the car.
If the maximum acceleration is M1 > 0 and maximum deceleration
is M2 > 0, then the controls satisfy the following constraints
0 ≤ u1 (t) ≤ M1
−M2 ≤ u2 (t) ≤ 0.


0 ≤ u1 (t) ≤ M1
−M2 ≤ u2 (t) ≤ 0.


0 ≤ u1 (t) ≤ M1
−M2 ≤ u2 (t) ≤ 0.


0 ≤ u1 (t) ≤ M1
−M2 ≤ u2 (t) ≤ 0.


0 ≤ u1 (t) ≤ M1
−M2 ≤ u2 (t) ≤ 0.

In addition, if the car starts with G liters of gas and there is no

service stations on the way, another constraint is given by
Z tf
[k1 u1 (t) + k2 x2 (t)] dt ≤ G
t0
which assumes that the rate of gas consumption is proportional to
both acceleration and speed with constants of proportionality k1
and k2 .

In addition, if the car starts with G liters of gas and there is no

service stations on the way, another constraint is given by
Z tf
[k1 u1 (t) + k2 x2 (t)] dt ≤ G
t0
which assumes that the rate of gas consumption is proportional to
both acceleration and speed with constants of proportionality k1
and k2 .

Denition (3. Admissible Control)

A control history which satises the control constraints during the
entire time interval [t0 , tf ] is called admissible control.
We shall denote the set of admissible controls by U , and the notation

u ∈ U means that the control (history) u is admissible.

Possible acceleration (control) histories: (Admissible or Not?)
a)
ADMISSIBLE
b)
NOT ADMISSIBLE
a)
ADMISSIBLE
b)
NOT ADMISSIBLE
a)
ADMISSIBLE
b)
NOT ADMISSIBLE
c)
ADMISSIBLE
d)
NOT ADMISSIBLE
NOTE: a) and c) are admissible given that they satisfy the fuel constraint.
c)
ADMISSIBLE
d)
NOT ADMISSIBLE
c)
ADMISSIBLE
d)
NOT ADMISSIBLE
Denition (4. Admissible Trajectory)

A state trajectory which satises the state variable constraints during
the entire time interval [t0 , tf ] is called an admissible trajectory.
The set of all admissible trajectories will be denoted by X , and x ∈ X

means that x is admissible.


Recall the following state constraint:

0 e
x(t0 ) =
0
=0 and x(tf ) =
0
0 ≤ x1 (t) ≤ e
0 ≤ x2 (t)
These constraints dictates the set of admissible trajectories X .

Now, the nal state of the system will be required to lie in a

specied region, say S , of the (n + 1)−dimensional state-time
space.
In our example (automobile problem), we have a 3-dimensional
state-time space with x1 , x2 , (states) and t (time) as axes.
We shall call S the target set.
For instance, if the nal state and the nal time are xed, S
is a point.


space.
is a point.


space.
is a point.


space.
is a point.


In our automobile problem, the target set S is a line .






Or, if the car is has been required to arrive within three feet of e
with zero terminal velocity, the target set S would have been:





Importance of Admissibility
Admissibility is an important concept in optimal control since
it reduces the range of values that can be assumed by the
states and controls.
Rather than considering all trajectories to nd the best
(according to some objective), we only investigate the
trajectories and controls that are admissible.

Importance of Admissibility
Admissibility is an important concept in optimal control since
it reduces the range of values that can be assumed by the
states and controls.
Rather than considering all trajectories to nd the best
(according to some objective), we only investigate the
trajectories and controls that are admissible.

The Performance Measure
In order to evaluate the performance of a system

quantitatively, the designer selects a performance measure.
An optimal control is dened as one that minimizes (or
maximizes) the performance measure.
In some cases, the problem statement may clearly indicate the
performance measure, whereas in other problems, the
selection of the performance measure can be subjective.
Example (Performance measure selection)

Problem statements:
1 "Transfer from point A to point B as quickly as possible."
2 "Maintain the position and velocity of the system near zero with
small expenditure of control energy."



Problem statements:



Problem statements:



Problem statements:



Problem statements:



Problem statements:


Problem statements:
The rst one clearly implies that the elapsed time is the
performance measure to be minimized.
On the other hand, the other statement does not suggest a unique
performance measure. In such cases, the designer may be required
to consider several performance measures before selecting the best
one that yields what he/she considers as the optimal control.


Problem statements:


Problem statements:


Problem statements:


Problem statements:


Selection of performance measure
Currently, we have a mathematical description of the automobile
problem and the physical constraints for the state variables and
control inputs.
Now, we are ready to selection the performance measure for our
system.
Suppose, the objective is to make the car reach point e as
fast as possible.
Then, the performance measure, say J , is given by
J = tf − t0


control inputs.
system.
fast as possible.
J = tf − t0


control inputs.
system.
fast as possible.
J = tf − t0


control inputs.
system.
fast as possible.
J = tf − t0


control inputs.
system.
fast as possible.
J = tf − t0

In general and for future discussions, we will assume that the
performance of any system J is evaluated by a measure of the form
R tf
J = h(x(tf ), tf ) + t0 g(x(t), u(t), t)dt
where t0 and tf are the initial and nal time, respectively; h and g
are real-valued functions. This is the general form of a
performance measure.
The nal time tf may be specied or "free", depending on the
problem statement.
In our automobile problem example, the performance measure can be
written as
R tf
J = tf − t0 = t0 dt

R tf
problem statement.
written as
R tf
J = tf − t0 = t0 dt

R tf
problem statement.
written as
R tf
J = tf − t0 = t0 dt

Starting from an initial state x(t0 ) = x0 and

applying a control u(t), for t ∈ [t0 , tf ], causes a
system to follow some state trajectory.
The performance measure assigns a unique
real number to each tra jectory of the system.

Starting from an initial state x(t0 ) = x0 and

applying a control u(t), for t ∈ [t0 , tf ], causes a
system to follow some state trajectory.
The performance measure assigns a unique
real number to each tra jectory of the system.

The Optimal Control Problem
THE OPTIMAL CONTROL PROBLEM

Find an admissible control u∗ which causes the system
ẋ(t) = a(x(t), u(t), t)
to follow an admissible trajectory x∗ that minimizes the performance

measure
Z tf
J = h(x(tf ), tf ) + g(x(t), u(t), t)dt.
t0
u∗ is called the optimal control and x∗ an optimal trajectory

THE OPTIMAL CONTROL PROBLEM

Z tf
MIN J = h(x(tf ), tf ) + g(x(t), u(t), t)dt.
t0
subject to
ẋ(t) = a(x(t), u(t), t)
x(0) = A intial condition
x(t) free
x(t) ∈ X admissible state trajectory
u(t) ∈ U admissible control

Some comments on the optimal control problem formulation:
Existence of solution is an issue but modellers usually attempt to

nd the optimal control rather than proving its existence.
Even if an optimal control exists, it may not be unique. Alternative
solutions can complicate computations but they do allow other
possibilities of choosing among several control congurations.
Minimization (or maximization) in an optimal control problem
refers to the search for the global optimum of J , not merely the
local optimum.
These statements mean that optimal control problems are
very similar to other optimization problems previously
discussed in the course.


local optimum.


local optimum.


local optimum.

A representation of an optimization (minimization) problem

Now, let us illustrate the complete formulation of our autmobile
problem and summarize the current results.
The state equations are
ẋ1 (t) = x2 (t)
ẋ2 (t) = u1 (t) + u2 (t).
The set of admissible states X is subjected to the following

conditions

e
x(t0 ) = 0 and x(tf ) =
0
0 ≤ x1 (t) ≤ e
0 ≤ x2 (t)

ẋ1 (t) = x2 (t)
ẋ2 (t) = u1 (t) + u2 (t).

conditions

e
x(t0 ) = 0 and x(tf ) =
0
0 ≤ x1 (t) ≤ e
0 ≤ x2 (t)

ẋ1 (t) = x2 (t)
ẋ2 (t) = u1 (t) + u2 (t).

conditions

e
x(t0 ) = 0 and x(tf ) =
0
0 ≤ x1 (t) ≤ e
0 ≤ x2 (t)

ẋ1 (t) = x2 (t)
ẋ2 (t) = u1 (t) + u2 (t).

conditions

e
x(t0 ) = 0 and x(tf ) =
0
0 ≤ x1 (t) ≤ e
0 ≤ x2 (t)


The set of admissible controls U is subjected to the following
constraints
0 ≤ u1 (t) ≤ M1
−M2 ≤ u2 (t) ≤ 0
In addition, both X and U are subjected to the inequality dened

by
Z tf
[k1 u1 (t) + k2 x2 (t)] ≤ G
t0
The objective is to minimize the performance measure J = tf − t0 .


constraints
0 ≤ u1 (t) ≤ M1
−M2 ≤ u2 (t) ≤ 0

by
Z tf
[k1 u1 (t) + k2 x2 (t)] ≤ G
t0


constraints
0 ≤ u1 (t) ≤ M1
−M2 ≤ u2 (t) ≤ 0

by
Z tf
[k1 u1 (t) + k2 x2 (t)] ≤ G
t0


constraints
0 ≤ u1 (t) ≤ M1
−M2 ≤ u2 (t) ≤ 0

by
Z tf
[k1 u1 (t) + k2 x2 (t)] ≤ G
t0


Assuming that M1 = M2 = M and that the car has enough fuel
available to reach point e, we have the following optimal control
and optimal trajectory.
Optimal controls


Optimal controls

Optimal controls


Optimal trajectories





Form of the Optimal Control
Denition (6. Optimal Policy)

If a functional relationship of the form
u∗ (t) = f (x(t), t)
can be found for the optimal control at time t, then the function f is
called the optimal control law, or the optimal policy.

Example (Optimal policy)

From the denition, we can say that f is a rule that assigns the optimal
control at time t for any admissible state value at time t. For example,
suppose we have the following optimal policy for a specic problem:
u∗ (t) = Fx(t)
Given that F is an n × m matrix of real numbers, then Fx(t) is

just a product of matrices.
Given this optimal policy, we can say that the optimal control is
linear, and time-invariant feedback of the states.


u∗ (t) = Fx(t)



u∗ (t) = Fx(t)


Denition (7. Open-loop and closed-loop control system)

If the optimal control is determined as a function of time for a specied
initial state value, that is,
u∗ (t) = e(x(t0 ), t)
the optimal control is said to be in open-loop form.
Thus, the optimal open-loop control is only optimal for a particular

initial state value, whereas, if the optimal control law is known
(closed-loop form), the optimal control history starting from any state
value can be generated.
A closed-loop optimal control is also called an optimal feedback control .

Closed-loop vs. Open-loop control systems
A system that maintains a prescribed relationship between the

output (function value of a state variable) and the reference
(control) input by comparing them and using the dierence as a
means of control is called a feedback/closed-loop control system.
Example (A feedback/closed-loop control system)
An example would be a air-conditioning control system. By measuring
the actual room temperature and comparing it with the reference
temperature (desired temperature), the thermostat turns the heating or
cooling equipment on or o in such a way as to ensure that the room
temperature remains at a comfortable level regardless of outside
conditions.

A system that maintains a prescribed relationship between the

output (function value of a state variable) and the reference
(control) input by comparing them and using the dierence as a
means of control is called a feedback/closed-loop control system.
Example (A feedback/closed-loop control system)
An example would be a air-conditioning control system. By measuring
the actual room temperature and comparing it with the reference
temperature (desired temperature), the thermostat turns the heating or
cooling equipment on or o in such a way as to ensure that the room
temperature remains at a comfortable level regardless of outside
conditions.

Those systems in which the output has no eect on the control

action are called open-loop control systems.
In other words, in an open-loop control system the output is
neither measured nor fed back for comparison with the input.
Example (An open-loop control system)
One practical example is a washing machine. Soaking, washing, and
rinsing in the washer operate on a time basis. The machine does not
measure the output signal, that is, the cleanliness of the clothes.





An advantage of the closed-loop control system is the fact that the

use of feedback makes the system response relatively insensitive to
external disturbances and internal variations in system parameters.
A disturbance is a signal that tends to adversely aect the value of
the output of a system.
From the point of view of stability, the open-loop control system is
more stable and reliable since feedback is ignored and the optimal
control is based only on an initial state value.





On the other hand, stability is a major problem in the closed-loop

control system, which may tend to over-correct errors and thereby
can cause oscillations of constant or changing amplitude.
For systems in which the inputs are known ahead of time and in
which there are no disturbances it is advisable to use open-loop
control.
Closed-loop control systems have advantages only when
unpredictable disturbances and/or unpredictable variations in
system components are present.
A proper combination of open-loop and closed-loop controls will
give satisfactory overall system performance.


control.


control.


control.

Optimal Control PDF

Transféré par

Informations du document

Description originale:

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Optimal Control PDF

Transféré par

Droits d'auteur :

Formats disponibles

Section 3 : Introduction to Optimal Control

November 21, 2017

To determine the control signals that will cause a

A control system can be any time-dependent process:

AMAT 112 Introduction to Optimization Modelling

To determine the control signals that will cause a

A control system can be any time-dependent process:

AMAT 112 Introduction to Optimization Modelling

To determine the control signals that will cause a

A control system can be any time-dependent process:

AMAT 112 Introduction to Optimization Modelling

1 A mathematical description (or model) of the process

AMAT 112 Introduction to Optimization Modelling

1 A mathematical description (or model) of the process

AMAT 112 Introduction to Optimization Modelling

1 A mathematical description (or model) of the process

AMAT 112 Introduction to Optimization Modelling

Suppose we have a process dened by the following:

u1 (t), u2 (t), . . . , un (t)

NOTE: xi and ui are functions of time t

AMAT 112 Introduction to Optimization Modelling

Suppose we have a process dened by the following:

u1 (t), u2 (t), . . . , un (t)

NOTE: xi and ui are functions of time t

AMAT 112 Introduction to Optimization Modelling

ẋ1 (t) = a1 (x1 (t), x2 (t), . . . , xn (t), t)

AMAT 112 Introduction to Optimization Modelling

AMAT 112 Introduction to Optimization Modelling

AMAT 112 Introduction to Optimization Modelling

AMAT 112 Introduction to Optimization Modelling

Example (Example of a control system)

Suppose we have a parked car to be driven in a straight line away from

AMAT 112 Introduction to Optimization Modelling

The resulting dierential equation that describes the change in

Selecting position and velocity as state variables, that is

The resulting dierential equation that describes the change in

Selecting position and velocity as state variables, that is

The resulting dierential equation that describes the change in

Selecting position and velocity as state variables, that is

We nd the state equations become

or using matrix notation, we have

Mathematical model of the process in state form

We nd the state equations become

or using matrix notation, we have

Mathematical model of the process in state form

We nd the state equations become

or using matrix notation, we have

Mathematical model of the process in state form

Suppose t ∈ [t0 , tf ], we have the following denition:

Denition (2. State Trajectory)

The terms "history", "curve", "function" and "trajectory" will be

AMAT 112 Introduction to Optimization Modelling

Suppose t ∈ [t0 , tf ], we have the following denition:

Denition (2. State Trajectory)

The terms "history", "curve", "function" and "trajectory" will be

AMAT 112 Introduction to Optimization Modelling

Suppose t ∈ [t0 , tf ], we have the following denition:

Denition (2. State Trajectory)

The terms "history", "curve", "function" and "trajectory" will be

AMAT 112 Introduction to Optimization Modelling

Suppose t ∈ [t0 , tf ], we have the following denition:

Denition (2. State Trajectory)

Suppose we have a process dened by the following:

Suppose we have a process dened by the following:

The resulting dierential equation that describes the change in

The resulting dierential equation that describes the change in

The resulting dierential equation that describes the change in

We nd the state equations become

We nd the state equations become

We nd the state equations become

Suppose t ∈ [t0 , tf ], we have the following denition:

Denition (2. State Trajectory)

Suppose t ∈ [t0 , tf ], we have the following denition:

Denition (2. State Trajectory)

Suppose t ∈ [t0 , tf ], we have the following denition:

Denition (2. State Trajectory)

Suppose t ∈ [t0 , tf ], we have the following denition:

Denition (2. State Trajectory)

It is more important to keep mind the dierence between a

The gure above shows a single-valued function of time which

It is more important to keep mind the dierence between a

The gure above shows a single-valued function of time which

It is more important to keep mind the dierence between a

The gure above shows a single-valued function of time which

It is more important to keep mind the dierence between a

The gure above shows a single-valued function of time which

First, let us dene the state constraints .

First, let us dene the state constraints .

First, let us dene the state constraints .

First, let us dene the state constraints .

Denition (3. Admissible Control)

Denition (4. Admissible Trajectory)

Now, the nal state of the system will be required to lie in a