Chapter 5

Nonlinear Programming

Chemical Engineering Department

National Tsing-Hua University
Prof. Shi-Shang Jang
May, 2003

1
Problem Statement

min f ( x)
s.t.g j ( x)  0
hk ( x)  0
xR N

j  1,..., M . k  1,..., K

2
 5-1 Examples
Example: Multistage Compressor
Optimization

Driver stage1 stage2 stage3

T T T T ,P=64atm

N (ml/hr) P = 1 (atm) X1 X2

3
Example: Multistage Compressor
Optimization
K 1

 K  Pout  K
 K 
W  NRT     NRT  
 K  1  in 
P  K  1 
Cp
K , R  idea gas constant
Cv
 

 K  x1 

 x2   64  
Wtotal  NRT           3
 K  1  1   x1   x2  
K 1
where  
K
1
let   , fixed T, N
4
we have :
4
Example: Multistage Compressor
Optimization

1/ 4 1/ 4
 x2   64 
min f  x   x
1/ 4
1      
 x1   x2 
s.t.x1  1
x2  x1
64  x2

5
5-3 Lagrange Multiplier
min f ( x)
s.t. h( x)  0

set L x, v   f  x   vhx 
find the necessary condition of the optimality of L(x, v)
f h
L 0 v  0see figure 
x x x
L  0  h x 
v
6
Lagrange Multiplier
f(x,y)=(x-3)2+(y-3)2,h(x)=y+x2-3x=0

7
Solution

L  x  3   y  3  v y+x -3x 
2 2 2

L  2(x-3 )-v( 2 x-3 )  0

x
L  2(y-3 )-v  0
y
L  y+x 2 -3x  0
v
x  2, y  2, v  2

8
Physical Meaning of Lagrange
Multiplier – Shadow Price

min f ( x)
s.t.h( x)  b
L( x, v)  f ( x)  vh( x)
L L
x*, v * solves  0,  ( h( x )  b)  0
x v
f
It can be shown that v*  x  x *,v  v*
b

9
5-4 Kuhn-Tucker Condition
min f ( x)
s.t.g j ( x)  0
hk ( x)  0
j  1, , M . k  1, , K

K  T Condition :
f   j u j g j  k vk hk  0N equations 
g j  0, hk  0
u j g j  0, u j  0

10
K-T Condition - Example

min f ( x)  x  x2
2
1

s.t.g1  x1  1  0
g 2  30  x  x  0
2
1
2
2

h1  x1  x2  6  0
x1*, x2 *  1, 5
11
K-T Condition - Example
2 x1  u1  2 x1u2  v1  0
 1  2 x2u2  v1  0
x1  1  0
30  x12  x22  0
x1  x2  6  0
u1  x1  1  0
 
u2 30  x12  x22  0
u1 , u2  0

12
K-T Condition - Example
 The above problem
is reduced to :

2  u1  2u2  v1  0
 1  10u2  v1  0
u2  0, v1  1, u1  3

13
K-T Theory
 Theorem 1: (Necessity) Consider the NLP
problem (1). Let f, g and h be differentiable
and x* is the solution, also gj(x*) and hk(x*)
be differentiable. Then there exist u*, v* solve
K-T problem.
 Theorem 2 : (Sufficiency)
 Consider the NLP problem (1). Let f(x) be
convex, g (x) be concave functions, and h(x)
be linear if there exist x*, u* and v* solve K-T
problem., then x* is the optimality of (1).

14
K-T Theory - Example
 f(x)=x12-x2 (1,1),(2,2),(1.5,1.5)
 f(1,1)=0, f(2,2)=2,f(1.5,1.5)=0.75<1
 ∴f convex
 h1=x1+x2-6 linear
 g1=x1-1 concave
 g2=30-x12-x22 (1,1), (2,2),(1.5,1.5)
 g2(1,1)=28, g2(2,2)=22, g2(1.5,1.5)=25.5>25,
 ∴g2 concave

 The solution is global optimum

15
5-5 Duality of Linear
Programming
 Primal problem:
n
min y   ci xi
i 1
n
s.t. a ji xi  b j  0; xi  0
i 1

 The corresponding Lagrangian can be

expressed by m
n
 n  n
L   ci xi    j   a ji xi  b j    i xi
i 1 j 1  i 1  i 1

16
 Duality of Linear Programming
 At x*
n
L*   ci xi*
i 1
 Also
L m
 ck    j a jk   k  0
xk j 1

 Substitute for ci above get:

n  m  m
 n
 m
L*   xi*   *j a ji   i    *j  b j   a ji xi*    *j b j
i 1  j 1  j 1  i 1  j 1

17
Duality of Linear Programming
 The Dual Problem hence becomes:
m
max w    j b j
j 1
m
s.t.  j a jk  ck
j 1

with j  0

18
Example – Primal Problem
max 60 x1  30 x2  20 x3
s.t.8 x1  6 x2  x3  48
4 x1  2 x2  1.5 x3  20
2 x1  1.5 x2  0.5 x3  8
x2  5
x1 , x2 , x3  0
19
Example – Dual Problem

min 481  202  83  54

s.t.81  42  23  60
61  22  1.53  4  30
1  1.52  0.53  20
1 , 2 , 3 , 4  0

20
Example: Charity Work
 The local college’s School of American Craftsman
has decided to participate seriously in a local charity
event by manufacturing special commemorative
desks, tables, and chairs. Resources are restricted
as shown in the next table. The local lumber
cooperative has donated 48 board feet of lumber.
Faculty and staff have volunteered 20 finishing hours
and 8 carpentry hours. The desk will be sold for $60,
the table $30 and the chair for $20. The school does
not expect to sell more than 5 tables. How does the
school should do to maximize the profit?

21
Example: Charity Work-
Continued
Resources Desk Table Chair

Lumber (Board feet) 8 6 1

Finishing hours (hours) 4 2 1.5

Carpentry (hours) 2 1.5 0.5

22
Example: Charity Work-
Continued (Primal and Dual)
 Primal: max z  60 x1  30 x 2  20 x3
s.t. g1 : 8 x1  6 x 2  x3  48
g 2 : 4 x1  2 x 2  1.5 x3  20
g 3 : 2 x1  1.5 x 2  0.5 x3  8
g 4 : x2  5
x1 , x 2 , x3  0

 Dual: min w  481  202  83  54

s.t. g 1 : 81  42  23  04  60
g 2 : 61  22  1.53  4  30
g 3 : 1  1.52  0.53  04  20
1 , 2 , 3 , 4  0

23
Example: Charity Work-
Continued (MATLAB Code)
%Primal
b=[48;20;8;5];
Aeq=[];beq=[];
UB=[Inf Inf Inf]';LB=[0 0 0]';
f=[-60;-30;-20];
A=[8 6 1;4 2 1.5;2 1.5 0.5;0 1 0];
[X,FVAL,EXITFLAG,OUTPUT,LAMBDA]=LINPROG(f,A,b,Aeq,beq,LB,UB);
X'
LAMBDA.ineqlin
%Dual
ff=f;
AA=-A';
bb=b;
UB=[Inf Inf Inf Inf]';LB=[0 0 0 0]';
[X,FVAL,EXITFLAG,OUTPUT,LAMBDA]=LINPROG(bb,AA,ff,Aeq,beq,LB,UB)
X'
LAMBDA.ineqlin

24
Example: Charity Work-
Continued (MATLAB Result)
 >> Optimization terminated successfully.

ans =
2.0000 0.0000 8.0000

ans =
0.0000
10.0000
10.0000
0.0000

Optimization terminated successfully.

ans =
0.0000 10.0000 10.0000 0.0000

ans =
2.0000
0.0000
8.0000

25
5-6 MATLAB NLP tool
 FMINCON Finds a constrained minimum of a function of several
variables.
 FMINCON solves problems of the form:
 min F(X) subject to: A*X <= B, Aeq*X = Beq (linear constraints)
 X C(X) <= 0, Ceq(X) = 0 (nonlinear constraints)
 LB <= X <= UB

 X=FMINCON(FUN,X0,A,B) starts at X0 and finds a minimum X to
the function
 FUN, subject to the linear inequalities A*X <= B. FUN accepts input
X and
 returns a scalar function value F evaluated at X. X0 may be a scalar,
 vector, or matrix.

26
MATLAB NLP tool-continued
 X=FMINCON(FUN,X0,A,B,Aeq,Beq) minimizes FUN subject to
the linear equalities
 Aeq*X = Beq as well as A*X <= B. (Set A=[] and B=[] if no
inequalities exist.)

 X=FMINCON(FUN,X0,A,B,Aeq,Beq,LB,UB) defines a set of
lower and upper
 bounds on the design variables, X, so that a solution is found
in
 the range LB <= X <= UB. Use empty matrices for LB and
UB
 if no bounds exist. Set LB(i) = -Inf if X(i) is unbounded below;
 set UB(i) = Inf if X(i) is unbounded above.

27
MATLAB NLP tool-continued
 X=FMINCON(FUN,X0,A,B,Aeq,Beq,LB,UB,NO
NLCON) subjects the minimization to the
 constraints defined in NONLCON. The
function NONLCON accepts X and returns
 the vectors C and Ceq, representing the
nonlinear inequalities and equalities
 respectively. FMINCON minimizes FUN
such that C(X)<=0 and Ceq(X)=0.
 (Set LB=[] and/or UB=[] if no bounds exist.)

28
Example: Multi-Stage
compressor (MATLAB Code)
A=[-1 0;1 -1;0 1];
B=[-1;0;64];
X0=[2;10];
X=FMINCON('stage_com',X0,A,B)

function obj=stage_com(x);
x1=x(1);x2=x(2);
obj=x1^(0.25)+(x2/x1)^0.25+(64/x2)^(0.25);

29
Example: Multi-Stage
compressor (MATLAB Result)
>> Warning: Large-scale (trust region) method does not currently solve
this type of problem,
switching to medium-scale (line search).
> In C:\MATLAB6p5\toolbox\optim\fmincon.m at line 213
In C:\MATLAB6p5\work\stage_com_main.m at line 4
Optimization terminated successfully:
Magnitude of directional derivative in search direction
less than 2*options.TolFun and maximum constraint violation
is less than options.TolCon
No Active Constraints

X=

3.9994
16.0011

30
Example

min f ( x)  x1  3  x2  3

2 2

s.t.g1  2 x1  x  1  0
2
2

g 2  9  0.8 x  2 x2  0
2
1

31
Contour Plot and Constraints

32
 MATLAB PROGRAM
 X0=[2 1];A=[];B=[];Aeq=[];Beq=[];LB=[0 0];UB=[5 5];
 X=FMINCON('examp_FUN',X0,A,B,Aeq,Beq,LB,UB,'exa
mp_NONLCON')
 function y=examp_FUN(x)
 y=(x(1)-3)^2+(x(2)-3)^2;
 function [c,ceq]=examp_NONLCON(x)
 ceq=[];
 c(1)=-2*x(1)+x(2)^2+1;
 c(2)=0.8*x(1)^2+2*x(2)-9;
 X=
2.5000 2.0000

33