CE74.

13 Water Resources Systems

Tutorial on Dynamic Programming
31.10.2012
By: I Putu Santikayasa
Dynamic Programming
Dynamic Programming is an algorithm design
method that can be used when the solution to
a problem may be viewed as the result of a
sequence of decisions
Sequence of decisions: decision are made
sequentially, one after another, based on the
state of the system
Example
To find a shortest path
S A B T
3
4
5
2 7
1
5 6
S T
13 2
B E
9
A D
4
C F
2
1
5
11
5
16
18
2
Maximize reservoir operation
Q
1,t
K
2
K
1
Q
2,t
R
1,t
S
1,t
S
2,t
R
2,t
t t t t t
L R Q S S
, 1 , 1 , 1 , 1 1 , 1
+ =
+
t t t t t t
L R R Q S S
, 2 , 2 , 1 , 2 , 2 1 , 2
+ + =
+
1 , 1
K S
t
<
2 , 2
K S
t
<
( )

+
=
T
t
t t
B B Maximize
1
, 2 , 1
Solution for DP problems
Bellmans Principle: An optimal policy (a set of
decisions) has the property that whatever the
initial state and initial decisions are, the
remaining decisions must constitute an
optimal policy with respect to the state
resulting from the initial decision.
State Si of the system at a stage i, one must
proceed optimally till the last stage,
irrespective of how one arrived at the stage Si.
Solution for DP problems
In serial multistage decision making problems,
the stage numbers may be assigned in
increasing order either in the backward or
forward direction
In both methods, we must define the state
variable, state transformation equation and
the recursive relation in every stage very
precisely
Backward Recursion
User 1 User 2 User 3
Stage 1
Stage 2
Stage 3
Forward Recursion
User 1 User 2 User 3
Stage 1
Stage 2
Stage 3
Example (forward recursion)
A pipeline is to be laid between node G and C
shown in the figure below. The pipeline can
pass only along the routes shown by solid
lines between intermediate nodes in the
figure. The distance between two nodes is
shown in the line joining the two nodes.
Obtain the shortest distance for the pipeline
using dynamic programming.
A B
C
D E
F
G H
I
13
12
12
5
18
7
10
18
13 15
11 6
Solution
Solution
The shortest route G-D-A-B-C
Distance = 42
Example (Backward recursion)
Inflow during four seasons to a reservoir with
storage capacity of 4 units are, respectively,
2,1,3, and 2 units. Only discrete values (0,1,2,
..) are considered for storage and release.
Overflows from the reservoir are also included
in the release. Reservoir storage at the
beginning of the year is 0 units. Release from
the reservoir during a season result in the
following benefit which are same for all the
four seasons.
Release Benefits
0 -100
1 250
2 320
3 480
4 520
5 520
6 410
7 120
Solution
R1=1
R2=1
R3=3
R4=3
-benefit= 1460
Thank you