CHAPTER I LINEAR PROGRAMMING PROBLEMS

Let A be an m x n matrix and let c be a vector in

n
. Then a linear programming problem has the form
LP:Min {cx : Ax b , x 0}.
Written in detail , this is
Minimize c
1
x
1
+ c
3
x
3
+....+ c
n
x
n

Subject to the conditions that
a
11
x
1
+ a
12
x
2
+. . . + a
1n
x
n
< b
1
a
21
x
1
+ a
22
x
2
+. . . + a
2n
x
n
< b
2

a
m1
x
1
+ a
n2
x
2
+. . . +a
mn
x
n
< b
m
x
1
, x
2
,...,x
n
0.
Here the coefficients a
ij
and the numbers c
i
are given and one is asked to find the the numbers x
i
and the value c
.
x.
The solutions x
i
often are not unique even when they exist. However, the value c
.
x is unique when it exists.
Actually, the symbol
Min {c.x : Ax b , x 0}
denotes a number, viz., the minimum. Since the minimal numbers is often less important than the values of the variables
which give the minimal value, we write the linear programming problem with the symbol LP meaning that we desire a
set of values x
1
, , x
n
satisfying the constraints and producing the minimum.
Different Presentations of Linear Programming Problems
There are several different presentations of the LP.
CANONICAL FORM; Min{cx : Ax b, x > 0}.
STANDARD FORM: Min {cx : Ax= b, x > 0}.
MIXED FORM: Min {cx : a
i
x b
i
for i= 1,2,...,p and
a
i
x = b
i
for i = p+1,..., m }.
For the present we ,consider only problems in standard form. All other problems can be converted to problems in
standard form by various manipulations. We need some definitions.
Definition. Let A be an m x n matrix. A point x* in
n
is said to be a feasible solution for the LP: Min{ cx :
" some conditions"} if x* satisfies the conditions. Otherwise, the point x* is said to be infeasible.
Definition. The function z = c
.
x is called the objective function and particular values of the objective function are
called objective values.
Definition. The vector c is called the cost coefficient.
Notation. Specific numerical or vector values are denoted by variable notation with an asterisk if it is necessary to
Chapter I Linear Programming Problems page 1
distinguish the actual values from the variables.
Definition. A point x* is said to be optimal for the LP: Min{cx : " some conditions"} if x* is feasible and if cx*
cy* for all other feasible solutions y*.
Definition. A linear programming problem is said to be feasible if it has at least one feasible solutions. A linear
programming problem with no feasible solutions is said to be infeasible. A feasible linear programm- ming is said to be
unbounded if it has no optimal solutions.
We now show the relationship between LP problems in canonical and standard form.
Theorem. Let A be an m x n matrix . Let
LP(1): Min{cx : Ax b, x 0},
and let
LP(2): Min{ (c;0
m
)
.
y : [A; I
m
]y = b, y 0}.
Here (c,0)=(c
1
,c
2
,...,c
n
;0,...,0) (m zeros) and

Block matrix [A; I] =
a
11
a
12
a
1n
1 0 0
a
21
a
22
a
2n
0 1 0

a
m1
a
m2
a
mn
0 0 1
Let be the map of
m+n
into
n
given by (x
1
, ... , x
m+n
) = (x
1
, ... , x
n
). Then the restriction of to the set of
feasible solutions of LP(2) is ( i) a one-one map or the set of feasible solutions of LP(2) onto the set of feasible solutions
of LP(1), (ii) a one-one map of the set of optimal solutions of LP(2) onto the set of optimal solutions of LP(1), and (iii)
c.(x*) = (c;0).x* for every x* in
m+n
.
Remark. We have converted the LP(1) in canonical form to the LP(2) in standard form by adding as m any extra
variables to LP(1) as there are rows in the matrix A. We have not introduced any new feasible or optimal soutions, and
we can read the feasible and optimal solutions of LP(1) from those of LP(2) simply by deleting the coordinate values
corresponding to the added extra variables. Presumably, we have made the LP(1) harder to solve if we plan to solve
LP(1) by first solving LP(2) and then projecting the solution to
n
because the problem LP(2) has more variables. The
new variables have no meaning in the original LP(1) but the optimal value of the objective function is the same in both
problems.
Proof. (i). Suppose y* is feasible for LP(2). Then y
i
* 0 for i=1,...,m+n. Then writing out the rows of Ay* = b,
we have that
a
i1
y
1
*

+ a
i2
y
2
*

+. .. + a
in
y
n
*

+ y
n+i
* = b
i
(i=1,...,m).
.
Since all the y
i
* are positive we have that
a
i1
y
1
*

+a
i2
y
2
* +...+ a
in
y
n
* b
i
for i= 1,...,m. Thus, (y
1
*,...,y
n
*) is feasible for LP(1). Thus, the map p sends the feasible solutions of LP(2) into the
feasible solutions of LP(1). Now let (y
1
*,...,y
n
*) be a feasible solution of LP(2). Then y
i
0 for i = 1,...,n and
b
i
- a
i1
y* - . . . -a
in
y* = y
n+i
* > 0 (i = 1,...,m)
so that (y
1
*,...,y
n+m
*) forms a feasible solution y* for LP(2) with p(y*) = (y
1
*,...,y
n
*). This means that is onto.
Finally, we show that p is one-to-one. Suppose that y and y' are feasible solutions of LP(2) with p(y) = p(y') i.e.,
with y
i
= y'
i
for all i = 1,...,n. But the remaining m coordinates n+1,..,n+m of y and y' are equal because
y
n+i
= b
i
- a
i1
y
1
- ... - a
in
y
n
= y'
n+i
for i = 1,...,m. Thus, all the coordinates of y and y' are equal. This means that p is one- to-one.
Chapter I Linear Programming Problems page 2
(ii) and (iii). The statement (iii) is clear by simply evaluating the dot product. Then from (i) and (iii) the statement (ii)
follows immediately. Q.E.D.
Definiton. The variables x
n+1
, x
n+2
, ... ,x
n+m
added in the previous example are called slack variables. When slack
variables are added to a LP problem, the original variables are called the decision variables.
Example. Consider the LP:
Max {c
.
x | Ax < b, x > 0}.
This can be turned into a canonical problem by noticing that the set of feasible and optimal solutions of the LP is exactly
the same as those for the canonical
LP:Min{(-c).x | Ax b, x > 0 }.
Notice that the objective values are not the same, i.e.,
Min{ (- c) x | Ax < b, x > 0 } = Max {c
.
x | Ax < b, x > 0}
For the present, we can suppress the minus sign for the minimum. Later when we study duality it will be important to
replace it.
Example. The LP : Min { cx | Ax > b, x > 0 } has the same set of feasible solutions and optimal solutions as the
LP : Min{ c x | (-A)x -b, x > 0}.
Example. The LP with the two endpoint conditions given by
LP : Min { c x ) | Ax = b, 0 x u}.
has the same set of feasible and optimal solutions as the LP :

Min c
T
x
A
I
n
x
b
u
, x 0
Example. We can write a standard LP as a canonical LP as follows:
Min { c.x | Ax = b, x 0 }
becomes

Min c
T
x
A
- A
x
b
- b
, x 0 .
Example. We can write LP(1):
Min { c. x | Ax = b, 0 x u }
in standard form by a series of steps. First write this LP in canonical form as LP(2) :

Min c
T
x
A
- A
x
b
- b
, x 0 .
Then we can use the previous example to adjust the positivity constraints to give the LP(3) :

Min c
T
x
A
- A
I
n
x
b
- b
u
, x 0
The three preceding LP's have the same set of feasible and optimal solutions. Putting the last LP(3) into standard form by
adding slack variables, we get the LP(4) :
Chapter I Linear Programming Problems page 3
Min{ (c,0)
T
y | By = d , y > 0 },
where B is the (2m+n) x (2m+2n) block matrix

B =
A I
m
0 0
- A 0 I
m
0
I
n
0 0 I
m
and
d = [b;-b; u]
T
.
Then the feasible solutions of LP(4) are in one-one correspondence with the feasible solutions of the LP(1) under the map
(that is stripping away the last 2m+n coordinates of the 2m+2n coordinates that comprise the the solutions of the
LP(4). The same is true about the optimal solutions of the LP(4) and the LP(1).
Example (Variables with Upper Bounds). The LP(1): Min{ cx | Ax = b, 0 x < u } of the preceding example can
be converted to an LP in standard form in a slightly different way. The LP(5):
Min {c x | Ax = b, x + y = u, x 0, y 0} ,
has a matrix representation as

Min (c; 0)
x
y
A 0
I
n
I
n
x
y

b
u
,
x
y
0
The map (x;y) = x is a one-one map of the set of feasible (respectively, optimal) solutions of LP(5) onto the set of
feasible (resp., optimal) solutions of LP(1).
CONCLUSION. For the time being we can restrict our attention to LP's in standard form. Of course, we are adding to the
difficulty of actually generating a numerical answer by converting an LP in canonical form to an LP in standard form by
adding slack variables and sometimes equations. In general, we try to keep the extra variables and equations to a
minimum. However, the standard form can be handled by methods of linear algebra. In some cases we actually use
routines developed for standard forms without actually writing down the standard form explicitly. Such a routine is used
for the preceding example of two-sided constraints. We shall develop this later.
Example. The following LP is in canonical form:
Minimize -3x
1
- 2x
2
- 4x
3

Subject to the constraints
x
1
+ x
2
+ 2 x
3
4
2 x
1
+ 3 x
3
5
2 x
1
+ x
2
+ 3 x
3
7
x
1
, x
2
, x
3
0.
Written in matrix form this becomes

Min (- 3, -2, - 4)
x
1
x
2
x
3
1 1 2
2 0 3
2 1 3
x
1
x
2
x
3

4
5
7
,
x
1
x
2
x
3
0 .
In symbolic form the LP reads
Min {cx | Ax b, x 0 }
where
Chapter I Linear Programming Problems page 4
A =

1 1 2
2 0 3
2 1 3
, b =

4
5
7
, c = ( - 3, - 2, - 4).
In standard form the problem becomes
Minimize -3x
1
- 2x
2
- 4x
3
Subject to the constraints
x1 + x2 + 2 x3 + x4 = 4
2 x
1 + 3 x3 + x5 = 5
2 x
1 + x2 + 3 x3 + x6 = 7
x
1
, x
2
, x
3
, x
4
, x
5
, x
6
0.
The standard matrix form of the LP is
Minimize (-3,-2,-4,0,0,0).(x
1
,x
2
,x
3
,x
4
,x
5
,x
6
)
Subject to the constraints

1 1 2 1 0 0
2 0 3 0 1 0
2 1 3 0 0 1
x
1
x
2
x
3
x
4
x
5
x
6
=
4
5
7
,
x
1
x
2
x
3
x
4
x
5
x
6
0
In the standard form, the variables x
1
,x
2
,x
3
are the decision variables and the variables x
4
, x
5
, x
6
are the slack
variables.
Example (Surplus Variables). There is another method of converting the linear programming problem
LP: Min {cx | Ax b, x 0}
into standard form. Here A is an m x n matrix as usual. Instead of changing the order of the inequalities by multiplying
by -1 as was done before, the conversion is accomplished by adding extra variables x
n+1
,...,x
m+n
with coefficients of -1
to produce the LP:
Minimize c
1
x
1
+ . . . + c
n
x
n
Subject to the constraints
a
11
x
1
+ . . . + a
1n
x
n
- x
n+1
= b
1
a
21
x
1
+ . . . + a
2n
x
n
- x
n+2
= b
2

a
m1
x
1
+ . . . + a
mn
x
n
- x
m+n
= b
m
with x
1
, x
2
, , x
m + n
0.
The extra variables x
n+1
,...,x
m+n
are called surplus variables. We do not pursue this extensively in these notes.
Chapter I Linear Programming Problems page 5
The Tableau
We can summarize the information in the Standard LP in a block form (i.e., in a matrix form) in the same way we
summarize the information in a system of linear equations. Recall how we write a system of linear equations as a
coefficient matrix. Here we write the LP as a tableau. This is similar to the coefficient matrix of a system of linear
equations. However, we need an extra row to hold the additional information represented by the cost coefficient and the
objective value. Whereas we needed an m x (n+1) matrix to represent the Ax = b where A is an m x n matrix (recall
the extra column on the right is the constant column filled by the column vector b), we now need an (m + 1) x (n + 1)
matrix to write the LP :Min {cx | Ax = b, x 0 }.
Example. For the previous example, the tableau would be
1 1 2 1 0 0 4
2 0 3 0 1 0 5
2 1 3 0 0 1 7
- 3 - 2 - 4 0 0 0 0
Note that we have filled the lower left corner for the present with 0.
Definition. The Tableau of the LP : Min {c.x | Ax = b, x 0 } is the (m+1) x (n+1) matrix

T =
a
11
a
12
a
1n
b
1

a
m1
a
m2
a
mn
b
1
c
1
c
2
c
n
0
Example (Free Variables). Let S be a subset of {1,2,...,n}. In the LP :
Min { cx | Ax = b, x
i
0 for i S}
the variables x
j
for j S have no positivity constraints. Such variables are called free variables. Two methods are used to
convert a problem of this type with free variables into a problem in standard form.
Method 1. Let x
j
be a free variable. Then x
j
must satisfy the equations
a
i1
x
1
+ .... + a
ij-1
x
j-1
+ a
ij
x
j
+ a
ij+1
x
j+1
+ ... + a
1n
x
n
= b
n
,
or
a
ij
x
j
= b - ( a
i1
x
1
+ .... + a
ij-1
x
j-1
+ a
ij+1
x
j+1
+ ... + a
1n
x
n
),
for all i = 1,...,m. If one of the coefficients a
ij
0, then we substitute the expression
x
j
= a
ij
-1
( b - ( a
i1
x
1
+ .... + a
ij-1
x
j-1
+ a
ij+1
x
j+1
+ ... + a
1n
x
n
))
for x
j
in all the equations amd in the expression c.x for the objective value. This eliminates x
j
in all the equations. We
continue to do this in succession with all the free variables until all are eliminated.
Suppose that there is an impediment to this,i.e., suppose that a
ij
= 0 for i = 1,...,m. Suppose further that there is
a feasible solution x* = (x
1
*,...,x
j-1
*,x
j
*,x
j+1
*,...,x
n
*) for the LP. Then the tuple x*(a) = (x
1
*,...,x
j-
1
*,a,x
j+1
*,...,x
n
*) will be a feasible solution of the LP for any arbitrary number a. This is true because a does not
appear in any of the equations and a has no posititvity constraints. But this means that the set of possible objective
values for the LP includes the set
{c.x*(a) | a } = { c
j
a

+ c
1
x
1
* + ... + c
j-1
x
j-1
* + c
j+1
x
j+1
* + ... + c
n
x
n
* | a }.
So the LP is unbounded (unless of course c
j
= 0 and no x
j
appears at all in the entire LP, a case that we can rule out from
Chapter I Linear Programming Problems page 6
the start) because a in the preceding expression can assume any value. So if a
ij
= 0 for all i, we can assume the LP is
unbounded.
Method II. Again let x
j
be a free variable for the LP(1) :
Min { c.x | Ax = b, x
i
0 for i S}.
Then let x
j
' and x
j
" be two new variables with x
j
=x
j
' - x
j
" and with
x
j
' 0 and x
j
" 0. More precisely, consider the LP(2) :
Min { c
1
x
1
+ ... + c
j-1
x
j-1
+ c
j
x
j
+ (-c
j
)x
j
+ c
j+1
x
j+1
+ ... + c
n
x
n
|
a
i1
x
1
+ .... + a
ij-1
x
j-1
+ a
ij
x
j
+ (-a
ij
)x
j
+ a
ij+1
x
j+1
+ ... + a
1n
x
n
= b
n
i = 1, 2, ..... , m, and
x
1
. , x
j-1
, x
j
, x
j
, x
j+1
, , x
n
0}.
Then the set feasible (resp. the optimal) solutions of LP(2) and the set of feasible solutions (resp., optimal solutions) of
LP(1) are related by a map of the form
( x
1
, , x
j-1
,x
j
, x
j
,x
j+1
, , x
n
) ( x
1
, , x
j-1
, x
j
- x
j
x
j+1
, , x
n
).
So a feasible (resp., optimal) solution of LP(2) will return a feasible (resp., optimal) solution of LP(1).
In general it is desirable to reduce every linear programming problem to the smallest possible size by eliminating
variables or equations. We present several examples of techniques to accomplish this in particular cases. The most
obvious reason for the elimination of variables is the enhancement of our ability to solve a particular problem with a
reasonable amount of effort.
Chapter I Linear Programming Problems page 7