Module # 14:

Basic concepts of Linear

Programming (LP), Non-
Linear Programming
(NLP),
Quadratic Programming
(QP)
14.1 Linear Programming
The idea where one uses symbols or say letters , or any other appropriate alphabet nomenclature
for some unknowns in equations to express some mathematical relationships, and then solve these
equations to find those values may be credited to the early Babylonians, Chinese, Greeks, Indians,
Iranians. The Arab scholar and mathematician Al-Khawarizmi, whose name is the etymological origin
of the word algebra may be considered to the first person who made a systematic study and analysis
of linear equations. Though the methodology of elimination as a tool to solve for linear equation has
its origin in Chinese and Indians scholars, yet the formalization of this concept goes to the works of
two Germans namely John Carl Friedrich Gauss (1777-1855) and Wilhelm Jordan (1842-1899), and
today is what we know as the Gauss-Jordon elimination method. The question which may occur in the
mind of any reader at this stage is what significance does this have with our whole scheme of study,
and in order to motivate the reader about the beauty of this method let us illustrate this with a simple
example.

Example # 14.1
Consider you as the marketing manager have two different types of tooth pastes to sell, and for our
convenience we name them 1 and 2 . Now the respective incomes per unit item from the two
products are 1 and 2 , and for ease of understanding and explanation one assumes that income is a
proxy for the difference of the selling price (SP) and the cost price (CP) of the respective products,
where by CP implies the cumulative cost incurred for making a unit product. Moreover consider the
input consumption of raw materials for both the type of toothpaste is fixed say and the utilization of
raw materials for per unit manufacturing of 1 and 2 are 1 and 2 respectively. Given this our
main concern is to decide on the optimal quantities of 1 and 2 produced such that one is able to
maximize the profit. Given this background for the problem, let us first denote the quantities of 1
and 2 produced as 1 and 2 , then the simple optimization problem function would be to
: 1 1 + 2 2 , given 1 1 + 2 2 . There are various mathods to solve this problem,
and that would be the main focus of this module, but before that we will discuss few important
assumptions which are applicable for this types of problems.

Thus the concept of Operations Research (OR) can be defined very simply as the scientific approach
to decision making, such that one is able to obtain the optimal decision for the system (e.g.,
production process and control (PPC), investment strategy for an investor, formulation of diet for
patients in a hospital, minimum distance travelled by a truck which must deliver goods at few fixed
location, but in some order or preference, etc.) under some set constraints/impediments which cannot
be violated or some rules which have to be adhered to. The main development of OR took place
during World War II (WW-II), and the father of OR can be considered as G. Dantzig. Other pioneers,

Page 2 of 31
who helped in developing this area, are, Koopmans, Russell A. Ackoff, Richard Karp, T. Magnanti,
etc. Now the question is, what are those important or vital steps which are to be considered in a macro
framework such that one can understand as well as solve the problem such that the goal or objective is
achieved. Before that we very briefly discuss the following schematic diagram (Figure 14.1) which
will motivate the reader on the variety of methods facets for optimization problem.

Optimization
Problem

Classical Heuristics
Methods

Simplex Algorithm
Genetic Algorithm
Steepest Descent
SQP
Newtons Method
ACO
Conjugate gradient
Simulated
Branch and Bound etc. Annealing etc.

Figure 14.1: Schematic disgram to show the variety of methods available to solve Optimization
problems

An optimization problems are made up of three basic ingredients (we will see what are these three
ingredients very soon), which are: (i) An objective function which we want to minimize or
maximize, for instance, in a manufacturing process, we might want to maximize the profit or
minimize the cost, or say in fitting an experimental data to a user-defined model, we might minimize
the total deviation of observed data from predictions based on the model, while in designing an
automobile panel, we might want to maximize its crash strength, (ii) a set of unknowns or variables
which affect the value of the objective function, say for example in the manufacturing problem, the
variables might include the amounts of different resources used or the time spent on each activity, or
else in fitting-the-data problem, the unknowns are the parameters that define the model, and in the
panel design problem the variables used can be taken as the shape and dimensions of the panel and
finally (iii) a set of constraints that allow the unknowns to take on certain values but exclude others,

Page 3 of 31
so for the manufacturing problem, it does not make sense to spend a negative amount of time on any
activity, so we constrain all the "time" variables to be non-negative, while in the panel design
problem, we would probably want to limit the weight of the product and to constrain its shape within
some desirable and feasible range.

So in simple terms the general goal of an optimization problem can be stated as the goal to find the
optimal combination of parameters, denoted as variables, which optimizes a given quantity, usually
denoted by the objective function, subject to some constraints under allowable range. Thus we have,
given a function : from some set to the real numbers we seek an element 0 in such that
(0 ) () for all in ("minimization") or such that (0 ) () for all in
("maximization") given the desired set of constraints.

Thus a general optimization problem can be stated as

() = (1 , 2 , . , )

s.t: ( ) 0 = 1, 2, . . ,
= 1, 2, . . ,

The objective function can be optimized using one of the techniques mentioned in the above figure, or
we can employ other methods also to solve the optimization problem. So one can add, that the
technique used for the optimization depends upon the properties of the objective function and the
constraints. Suppose the objective function is multimodal, such that there is one global maximum and
many local maxima. In this case we cannot apply the traditional methods to solve the optimization
problem because it may get struck to local maxima and we may not be able to reach the global
maximum.
For e.g., consider a function f = sin(x) + cos(x) + tan(x) (Figure 14.2) with the constraint that < x
<, then it is obvious that that the function has two local maxima but only one global maximum.
Therefore, traditional methods to optimize this function may not be suitable for this problem as they
may get stuck to some local points. In such cases we resort to heuristics and meta heuristics which try
to find out the global maxima/minima (as the case may be), and these techniques would just be named
here, without going into how these meta-heuristics techniques work.

Page 4 of 31
 Figure 14.2: Plot of the function sin(x)+cos(x)+tan(x)

We know that the techniques for optimization depend upon other properties of objective function and
constraints (such as whether they are convex or concave), number of variables in the problem, etc. For
e.g., if we have an objective function which is concave then, we cannot apply any of the classical
methods (such as simplex algorithm, we will discuss about simplex method within few minutes) of
optimization to such an objective function. In such cases we have to resort to the meta heuristics for
optimizing the objective function, so there is no fixed rule as to which technique is to be used for
optimization given a particular problem and in many cases it comes by experience to know which of
the technique is to be used to solve the problem. But it is also imperative to mention here that the
formulation, i.e., how the optimization has been formulated also dictates the general solution
methodology employed, and this science of formulating the optimization problem comes from
experience and is more of an art than science.

14.2 Steps for mathematical formulation of a Linear Programming Model

1) Study the given situation, find the key decision to be made, whether it is maximization or
minimization. Hence, identify the decision variables of the problem.
2) Formulate the objective function to be optimized (whether maximization or minimization).
3) Formulate the constraints of the problem.
4) Add non-negativity restrictions.

14.2.1 Basic seven steps in Model building

1) Formulate the problem
2) Observe the system
3) Formulate a mathematical model for the problem
4) Verify the model and use the model for predictions

Page 5 of 31
5) Select a suitable alternative
6) Present the results
7) Implement and evaluate the recommendations

Example # 14.2
A company in its despatch shop fills up the cartons with either 100 gms or 500 gms jam bottles, and
there are two different automatic cartons filling up machines which does this job of filling up the
empty cartons. The machines are denoted as X and Y, and each of these X and Y machines can fill up
the cartons with either 100 gms or 500 gms bottles. The carton filling rates are given below
Machine 100 gms bottle 500 gms bottle
X 80 per 5 minutes 30 per 5 minutes
Y 40 per 5 minutes 50 per 5 minute
X and Y can run for 8 hours per shift of the day and for 5 days a week. Profit for 100 gms bottle is Rs.
10, while for 500gms it is Rs. 40 respectively. The weekly filling up rate cannot exceed 500000 gms
in total (i.e., combining the weights of the 100 gms and 500 gms bottles) due to storage capacity in the
warehouse from where these bottles filled up cartons are dispatched to different distributors. Also the
market has limitation on the demand and a marketing survey of the demand of these two types of
bottles have shown that maximum/minimum demands are 2500 and 500 respectively for 100 gms and
500 gms bottles respectively. What is the companys maximum profit?

Step # 1: Let x1 and x2 denote the number of 100 gms and 500 gms bottles which are filled up and
dispatched per week to the distributors.
Step # 2: The objective is to maximize profit which is given by (10*x1 + 40*x2)
Step # 3:
Constraint # 1: x1/80 + x2/30 8*12*5
Constraint # 2: x1/40 + x2/50 8*12*5
Constraint # 3: 100*x1 + 500*x2 500000
Constraint # 4a: x1 2500 and x2 500
Constraint # 4b: x1 2500 and x2 500

So the LP is as follows
Max z (10*x1 + 40*x2)
s.t: x1/80 + x2/30 8*12*5
x1/40 + x2/50 8*12*5
100*x1 + 500*x2 500000
x1 2500 and x2 500

Page 6 of 31
i.e.,
Max z (10*x1 + 40*x2)
s.t: x1/80 + x2/30 8*12*5
x1/40 + x2/50 8*12*5
100*x1 + 500*x2 500000
x1 2500 and x2 500

14.2.2 Broad classification of OR models

1) Static and Dynamic Models
2) Linear and Non-Linear Models
3) Integer and Non-Integer Models
4) Deterministic and Stochastic Models

Example # 14.3
A paint company manufactures two different types of paints, P1 and P2, from the raw materials, M1
and M2. The following table provide the basic data for our example
Tons of raw materials per ton of Maximum daily
P1 P2 availability (tons)
Raw material M1 6 4 24
Raw material M2 1 2 6
Profit per ton ($ 1000) 5 4
A marketing survey indicates that the daily demand for P2 cannot exceed that of P1 by more than 1
ton. Also the maximum daily demand of P2 is 2 tons. The company would like to determine the
optimal mix of the two paints in order to maximize its daily profit.
Step 1: To determine the amount to be produced of P1 and P2 we denote x1 as the amount of P1 paint
and x2 as the amount of P2 paint.
Step 2: To construct the objective function the company wants to increase its profit as much as
possible. If we denote the profit function as z, then we need to maximize z = 5*x1 + 4*x2.
Step 3: The constraints, that restricts the raw materials and demand is related by the fundamental
principle that {usage of raw materials for both the paints} {maximum raw materials available}
Thus we have from the table we have the following constraints:
6*x1 + 4*x2 24 1*x1 + 2*x2 6 1 + 2 1 2 2
With the problem as given, the schematic diagram (Figure 14.3) will make one understand the
pictorial representation for the two dimentional case. In case we have the problem in higher
dimension it may become difficult to represent the problem, but the general philosophy of pictorial
illustration will remains the same.

Page 7 of 31
Maximize = 51 + 42
s.t: 61 + 42 24 (14.1a)
11 + 22 6 (14.1b)
1 + 2 1 (14.1c)
01 + 2 2 (14.1d)
x1, x2 0 (14.1e)

Here the area A-B-C-D-E-A is the common area considered if one looks at the constraints and our
task if to find some point (x*1, x*2) such that we are able to maximize the objective function.

x2

1a

1b 1c
A B
1d C
C 1e
E D 1e
x1

Figure 14.3: Schematic diagram of the optimization problem

14.3 Few relevant concepts of Linear Algebra

In this section we will cover some relevant portions of linear algebra which are relevant to understand
how optimization works. The topics which we will cover are as follows
1) Vector
2) Scalar product of vectors
3) Martrix
4) Rank of a matrix
5) Addition of matrices
6) Determinant of a matrix
7) Transpose of a matrix
8) Inverse of a matrix

Page 8 of 31
9) Basis of a matrix
10) Multiplication of matrices
11) Gauss Jordan method
12) Convex set/function
13) Concave set/Function
14) Hyper plane
15) Hyper space

14.3.1 Vector
It is a one dimensional array and in general signifies to have both unit as well as some director. In the
mathematical sense as used in Operations Research we see that a vector as an array and is denoted by
a column or a row as shown below. Suppose we have a 3-D plot and a point is represented by the co-
ordinates x=10, y=25, z=5, then in vecore notation it will be represented as = (10,25,5), else it can
10
also be represented as = 25.
5

14.3.2 Scalar Product of Vectors

When we have two vecore then multiplication is possible provided there representation is done in
such a way that there dimension as well as their size. Thus given two vectors = (10,25,5) and
10 10 10
= 25, we can represent = (10,25,5) 25 = 750. In case we have = 25
5 5 5
(10,25,5) . Thus it is not commucative.

14.3.3 Matrix
A matrix (for which the plural is matrices) is a rectangular arrangment of numbers, symbols,
equations, etc., arranged in rows and columns. The individual items in a matrix and the numbers,
symbols, equations, etc., are called entries or elements of the matrix. The nomenclature of defining
the matrix is by defining the number of rows and columns, i.e., in case we have a matrix of size 3 3,
10 3 0 10
the matrix then an example of it can be as 25 15 5 . Thus the first colums is 25, while the
5 5 100 5

last row is (5 5 100). Hence the 2 row and 3 element is 5, which gives us the idea that the
elements are generally identifies as , , suh that 6 coulmn will be the set of elements which are
1,6 , 2,6 , , ,6 provided we have number of rows. Similarly the 14 row would consist of
elements which are 14,1 , 14,2 , , 14, when one considers number of coulms.

14.3.4 Rank of a Matrix

Page 9 of 31
 1,1 1,
The rank of a matrix = signifies the system of linear equations and linear
,1 ,
transformation encoded by A. In general it is the largest collection of linearly independent
columns/rows of , i.e., the maximum number of rows/columns which are independent and none of
these can be explained as a linear combination of the other rows/columns. The rank fo a marix is
5 10 0
denoted by rank(A). For example suppose = 4 8 5 . We can easily discern that 2 = 2
9 18 100
10 3 0
1 , thus the rank would not be 3 here, but for the case when = 25 15 5 , then the rank is the
5 5 100
maximum number of rows/columns, hence in this rank(B)=3.

14.3.5 Additional of Matrices

5 10 0 10 3 0
Suppose we have two matrices = 4 8 5 and = 25 15 5 , then the sum of and
9 18 100 5 5 100
5 10 0 10 3 0 5 + 10 10 + 3 0+0
is given as + = 4 8 5 + 25 15 5 = 4 + 25 8 + 15 5 5 =
9 18 100 5 5 100 9 + 5 18 + 5 100 + 100
15 13 0
29 23 10. Thus one can easily understand that + = + . Moreover this addition is
14 23 200
general is possible when numbers of rows and columns of the two matrices are equal.

14.3.6 Determinant of a Matrix

The concept of determinant can be associated with a square matrix, where when writing determinant
5 10 0
we use the parenthesis as follows. Suppose = 4 8 5 , then the determinant is written as
9 18 100
5 10 0
4 8 5 and the value associated with the determinant is given as () = 5{(8 100)
9 18 100
(5 18)} 10{(4 100) (5 9)} + 0{(4 18) (8 9)}. One should remember that the
determinant is generally associated wih a square matrix only.
Few interesting properties of determinant are as follows
1) ( ) = 1, where In is a identity matrix.
2) ( ) = ().
1
3) (1 ) = .
()
4) For matrices A and B which are square and of equal size, in the sense that the number of rows
of both A and B are equal, as is the number columns of them. The () = ()
().
5) () = (), where c is a constant and A is a matrix.

Page 10 of 31
14.3.7 Transpose of a Matrix
5 10 0 5 4 9
Given a matrix = 4 8 5 , its transpose is given as = 10 8 18 . Few relevant
9 18 100 0 1 100
properties of transpose are:

1) ( ) = = .
2) ( + ) = + .
3) () = .
4) () = , where is a scalar number.
5) ( ) = ().
6) ( )1 = (1 ) .

14.3.8 Inverse of a Matrix

Inverse of a matrix is denoted as (), such that () = () = . Here is an
identity matrix whose size, i.r., the number of rows and colums is same as . One should remember
that a matrix is invertible if the the determinant of the matrix exists. If we have the matrix =
1,1 1, 1,1 ,1
1
, then the inverse of given by , such that =
1 1
()
,
,1 , 1, ,
2,2 2. 5 8 9
where 1,1 = and so on. Let us consider an example where = 4 6 5. Then
2, , 2 10 11
() = 5(66 + 50) 8(44 10) + 9(40 12) = 776. Thus
0.1495 0.0026 0.1211
1 = 0.0438 0.0941 0.0786 . To corroborate the fact that 1 = 1 = , let us find
0.0670 0.08510.0026
the value of 1 as well as 1 . Hence
0.1495 0.0026 0.1211 5 8 9 5 8 9
1 = 0.0438 0.0941 0.0786 4 6 5 = 1
= 4 6 5
0.0670 0.08510.0026 2 10 11 2 10 11
0.1495 0.0026 0.1211 1 00
0.0438 0.0941 0.0786 = 0 1 0 .
0.0670 0.08510.0026 0 01

14.3.9 Basis of a Matrix

A basis is a set of linearly independent vectors that, using linear combination, can represent every
vector in a given vector space. Thus it defines a coordinate system such that all the vectors are
expressed using the basis combination. In more general terms, a basis is a linearly independent
spanning set. Hence if we are given a basis of a vector space, every element of the vector space can be
expressed uniquely as a finite linear combination of basis vectors. When forming the combination we

Page 11 of 31
need coefficients (referred to as vector components) which gives the relationship of the combination.
1 0 0
In the 3-D case, i.e., the Cartesian co-ordinate system, if 1
= 0, 2 = 1 and 3 = 0, then
0 0 1
anh vector or co-ordinate can be expresses as a combination of 1 , 2 and 3 . Hence a point (2,4,3) is
1 0 0
actually formed as follows, i.e., 2 0 + 4 1 +3 0 = 21 + 42 + 33 . This gives an example
0 0 1
of what we mean by a basis.

14.3.10 Multiplication of Matrices

5 8 9
Matrix multiplication is not commucative. For example consider = 4 6 5, while =
2 10 11
1 2 0 44 118 83 3 4 19
6 16 7 . Then = 45 88 27 , while = 80 214 51 , which shows that
1 0 3 47 164 103 11 22 24
. For matrix multiplication the concept used for multiplication is very simple with one
important thing to remember which is the fact of the size of the matrix.Suppose A and B are matrices
of size and respectively. When we pre-multiply B with A we have a matrix of size
, but when we try to post multiple B with A, then we see there is a mismatch of the size hence one
may conclude that does not exist.

14.3.11 Gauss Jordan Method

Gauss Jordian method is used to solve the set of simultaneous equation in variables. Let us
illustrate this with a simple example. Consider we have simultaneous equation in variables, and
they are represented as = , where A is a matrix, while X and B are vectors. Thus if we pre-
multiple A with 1 , provided determinant of exists, then we have 1 = 1 , i.e., =
1 . Solving this gives us the values of .

14.3.12 Convex Set/Function

An object is said to be convex if for every pair of points within the object, every point on the straight
line segment that joins them is also within the object. For example, a square is a convex set.while on
the other hand a a crescent shape, is not convex set. On the other hand a function is said to be convex
iff if its epigraph is a convex set. For the case of LP problem solving we must remember that the
solution/feasible space should be convex in nature.Few illustrations of convex set are given in Figure
14.4 and they convey the concept for the 2-D case. Similarly one can find examples for the 3-D and
higher dimensional case.

Page 12 of 31
 Figure 14.4: Convex set in 2-D
14.3.13 Concave Set/Function
Concave function is the negative of a convex function. Thus a function () is concave over a convex
set iff () is a convex function over the set. As already mentioned, for the case of LP problem, in
order to have solution/feasible space the set or function over which one would work is convex in
nature and not concave. For the interested reader we give few simple examples of concave set, and
Figure 14.5 shows examples of concave set in 2-D.

Figure 14.5: Concave set in 2-D

14.3.14 Hyper plane

A simple definition of a hyperplane would be, it is a generalization of the two-dimensional plane into
a larger number of dimensions. The hyperplane may be a Euclidean space or more generally an affine
space, or a vector space or a projective space, and the notion of hyperplane varies correspondingly.
Let 1 , 2 , , be equal to scalars not all equal to zero, then the set consisting of all vectors
[1 , 2 , . , ] in such that 1 1 + 2 2 + + = , where is a constant is a sub-space
of . Let us illustrate this with a diagram, Figure 14.6. Consider we have 1 1 + 2 2 + 3 3 = 1,
along with the condition that 1 , 2 and 3 are [0,1] such that the three coordinates = (1,0,0),
= (0,1,0) and = (0,0,1) are the corner points of the hyper plane as shown.

Page 13 of 31
 z

C
y
B
x
Figure 14.6: Hyper plane when we have the space as 3D

14.3.15 Hyper Space

When we talk about dimension we define it is the minimum number of coordinates needed to specify
any point within it. Thus a square is of 2-D, while on the other hand the the cube, a cylinder or a
sphere are all 3-D. Extending this anology if we have an n-D space then the space is called a hyper
space. Continuing with the logic, the dimension of a vector space is the number of vectors in any basis
for the space, i.e. the number of coordinates necessary to specify any vector. This notion of dimension
(the cardinality of a basis) is often referred to as the Hamel dimension or algebraic dimension to
distinguish it from other notions of dimension.

14.4 Properties of Linear Programming

In our discussion of Linear Programming (LP) we will assume the following properties to be true
when we solve any linear programming problem. Those relevant assumptions are as follows:
1) Proportionality: Under this assumption we consider that if the objective function = and
the usage of is given by , i.e., both the quantities are directly proportion to the level of
each activity , ( = 1, . , ), then it true that there is no extra set-up charge associated with the
beginning of the activity. Moreover proportionality holds over the entire range.
2) Additivity: Given any activity levels (1 , 2 , . , ) the total usage of each resource and their
total effectiveness is equal to the sum of the individual effectivesness of each resource when they
are considered individually.
3) Divisibility: If one considers this assumption then it means that eac activity and resources may
be divided into fractional units. This means that fractional units of decision variables are
permissible, though it may may not be true in actual practice.
4) Certainty/Deterministic: When we say that we consider the assumption of certainty to be true it
implies that all the parameters in the model are known constants

Page 14 of 31
14.5 Few important definition for Linear Programming
1) Feasible set: This is a solution for which all the constraints are satisfied. For example the area A-
B-C-A (Figure 14.7) is the feasible set and any one point in that is the optimal soltuion

Obj Fun.
C

B
A

Figure 14.7: Feasible set A-B-C-A for the optimization problem

2) Optimal Solution: If one refers to Figure 14.8 then the optimal solution is a feasible solution that
has the most favourable value of the objective function, be it maximum or minimum.

Obj Fun.
C

B
A

Figure 14.8: Optimal solution at C for the maximization problem

3) Unbounded solution: The concept of unbounded solution states that as we continue with the
optimization programming for the LP the optimal solution keeps increasing/decreasing with each
iteration as the case may be depending on whether one is solving the maximization/minimization
problem. Thus in Figure 14.9 we encounter such a case when finding the maximization. As the
objective function continues to improve with each iteration we see that it attains a value of
infinity, which in technical term means we are able to obtain a feasible solution that is infinite.
For the ease of understanding we should note how the objective function keeps on increasing
reaching a value of infinity in the long run.

Page 15 of 31
 Obj Fun.

Figure 14.9: Unbounded solution for the maximization peoblem

4) Multiple Solutution: When solving the optimization problem we may not get an optimal or
unique solution. In that case we obtan multiple solution such that none of them is either worse or
better than the other solution. For example in Figure 14.10 all the points along the line A-E are
optimal feasible solution as can be visualized by the fact that when the objective asses through
the boundary E-A it crosses each and every point at the same time, thus making all of them
optimal solution. Hence one of them are unique.
5) Shadow Prices: Assuming there are no other changes to the input parameters, the change to the
objective function value per unit increase to a right hand side of a constraint is called the Shadow
Price.
6) Sunk costs: The shadow price is the value of an extra unit of the resource, since the cost of the
resource is not included in the calculation of the objective function coefficient.
7) Included costs: The shadow price is the premium value above the existing unit value for the
resource, since the cost of the resource is included in the calculation of the objective function
coefficient.

Page 16 of 31
 Obj Fun.
E
D

B A

Figure 14.10: Infinte solution along A-E

14.6 Simplex Algorithm

After having discussed the general assumptions based on which LP can be solved we state below the
steps taken for solving the simplex method, wich is one of the most powerful method for solving the
linear programming problem. Thus the general plan of solving the simplex solution is as follows:
Step # 1: Determine the starting basic feasible solution
Step # 2: Select an entering variable using the conditional of optimality (would be different for
maximization and minimization problems)
Step # 3: Select the exiting variable
Step # 4: Determine the new basic feasible solutions and start the process again till no improvement
of the objective function is obtained.

Note: Remember the row and column operations are performed using the simple concept of Gauss-
Jordan principle. Let us again refer back to the simple diagram (Figure 14.11) which shows how the
we have already discussed before, though we may have done that very briefly.

Page 17 of 31
 Obj Fun. D

B
A

Figure 14.11: Pictorial representation of simplex methodology search for the optimal solution

Example # 14.4
Mayank Singh who is a DJ and the group leader of Halla Gulla music club is very health conscious
about himself and his group members. Remember it is a large music group with many members and
all the group members follow a strict diet regime. They are only allowed to take special variety of
mixed salad and a specially prepared paneer kofta as advised by their nutritionist Arjun Ravindra
Khular. They use at least 800 kgs of this combined special food daily. Yes we do agree the amount is
huge but remember the group members like to eat. The nutrient contents and the costs are given below
in the chart.

Kg per kg of food
Type of food Protein Fibre Cost (Rs.)
Mixed salad 0.09 0.02 0.30
Paneer kofta 0.60 0.06 0.90

The dietary requirement of this special type of food entails an intake of at least 30% protein and at
most 5% fibre. Solve the problem and help Arjun (who has no clue of how to solve an optimization
problem) such that the aim to minimize the total cost is met.

Let the decision variables be:

1 : Salad
2 : Paneer kofta
Hence the optimization problem is as follows
min . + . (14.2a)

Page 18 of 31
s.t.: + (14.2b)
. + . . ( + ) (14.2c1)
. . (14.2c2)
. + . . ( + ) (14.2d1)
. . (14.2d2)
, (14.2e1, 14.2e2)

(Eqn: 14.2e1)

(Eqn: 14.2d)
2
(Eqn: 14.2a)
A
(Eqn: 14.2c)

B (Eqn: 14.2b)
Eqn: 14.2e2)
0,0 1

Figure 14.12: Pictorial illustration for Example 14.5

Find points A and B and find the objective functions at A and B and then find the minimum value
For A: coordinate is (200,600), for which the objective function is 600

For B: coordinate is (470.6,329.4), for which the objective function is 437.64

Hence best possible combination is: (470.6,329.4), = 437.64

Example # 14.5
Sumit Kumar is a high profile person who has just become the CEO of Ghotala Bank Inc. in India,
after completing his MBA from Roopnagar University with flying colours. He is in the process of
devising a loan policy for his bank and the amount involves a maximum of Rs.120 crores. The
following table provides the pertinent data about the available types of loans.
Type of Loan Interest Rate Bad Debt Ratio
Personal 0.140 0.10
Car 0.130 0.07
Home 0.120 0.03

Page 19 of 31
Farm 0.125 0.05
Commercial 0.100 0.02
Note
Bad debts are unrecoverable and produce no interest revenue.

Competition with other banks requires that Ghotala Bank Inc. allocates at least 40% of the

funds to farm and commercial loans.

To assist housing industry in the country home loans must equal at least 50% of the personal,

car and home loans.

The bank also has a stated policy of not allowing the overall ratio of bad debts on all loans to

exceed 4%.

Help Sumit to formulate the optimization problem, given the data.

Let the decision variables be:
1 : Personal loans
2 : Car loans
3 : Home loans
4 : Farm loans
5 : Commercial loans
Given these we have the optimization problem is such that where we maximize difference between
total interest and bad debts, where
Total interest is
= 0.14 0.91 + 0.13 0.932 + 0.12 0.973 + 0.125 0.954 + 0.1 0.985
= 0.1261 + 0.12092 + 0.11643 + 0.118754 + 0.0985
Bad debts is
= 0.11 + 0.072 + 0.033 + 0.054 + 0.025
Hence objective function is total interest bad debts, hence we have
Maximize 0.0261 + 0.05092 + 0.08643 + 0.068754 + 0.0785
s.t.:
1 + 2 + 3 + 4 + 5 12 (14.3a)

4 + 5 0.4(1 + 2 + 3 + 4 + 5 ) (14.3b1)
0.41 + 0.42 + 0.43 0.64 0.65 0 (14.3b2)

3 0.5(1 + 2 + 3 ) (14.3c1)
0.51 + 0.52 0.53 0 (14.3c2)

Page 20 of 31
0.11 + 0.072 + 0.033 + 0.054 + 0.025 0.4(1 + 2 + 3 + 4 + 5 ) (14.3d1)
0.061 + 0.032 0.013 + 0.014 + 0.025 0 (14.3d2)

1 , 2 , 3 , 4 , 5 0 (14.3e)

Example # 14.6
Harpreet Singh & Associates produces both interior and exterior paints from two raw materials M1
and M2. The following table provides the basic data for the problem.
Raw Material Tons of raw material per ton of Maximum daily
Exterior Paint Interior Paint availability (tons)
M1 6 4 24
M2 1 2 6
Profit per ton ( 1000) 5 4
A marketing survey has revealed that the daily demand for interior paint cannot exceed that of
exterior paint by more than 1 ton, also the maximum daily demand of interior paint is 2 tons. Help
Harpreet Singh & Associates to solve this optimization problem using simple method (tableau based
solution only).
The starting tableau is as give
Step # 1
Basic z x1 x2 s1 s2 s3 s4 Solution
Z 1 -5 -4 0 0 0 0 0
s1 0 6 4 1 0 0 0 24
s2 0 1 2 0 1 0 0 6
s3 0 -1 1 0 0 1 0 1
s4 0 0 1 0 0 0 1 2
Thus the calculations yields
2 1 2 5
(1, 5, 4,0,0,0,0,0) (5) 0,1, , , 0,0,0,4 = 1,0, , , 0,0,0,20
3 6 3 6
1 2 1
(0,6,4,1,0,0,0,24) = 0,1, , , 0,0,0,4
6 3 6
2 1 4 1
(0,1,2,0,1,0,0,6) 1 0,1, , , 0,0,0,4 = 0,0, , , 1,0,0,2
3 6 3 6
2 1 5 1
(0, 1,1,0,0,1,0,1) (1) 0,1, , , 0,0,0,4 = 0,0, , , 0,1,0,5
3 6 3 6
2 1
(0,0,1,0,0,0,1,2) (0) 0,1, , , 0,0,0,4 = (0,0,1,0,0,0,1,2)
3 6

These calculations would be used for the 2nd tableau which is given below
Step # 2
Basic z x1 x2 s1 s2 s3 s4 Solution

Page 21 of 31
 z 1 0 -2/3 5/6 0 0 0 20
x1 0 1 2/3 1/6 0 0 0 4
s2 0 0 4/3 -1/6 1 0 0 2
s3 0 0 5/3 1/6 0 1 0 5
s4 0 0 1 0 0 0 1 2
Thus the calculations yields
2 5 2 1 3 3 3 1
1,0, , , 0,0,0,20 0,0,1, , , 0,0, = 1,0,0, , , 0,0,21
3 6 3 8 4 2 4 2
2 1 2 1 3 3 1 1
0,1, , , 0,0,0,4 0,0,1, , , 0,0, = 0,1,0, , , 0,0,
3 6 3 8 4 2 4 2
4 1 3 1 3 3
0,0, 3 , 6 , 1,0,0,2 4 = 0,0,1, 8 , 4 , 0,0, 2
5 1 5 1 3 3 3 5 5
0,0, , , 0,1,0,5 0,0,1, , , 0,0, = 0,0,0, , , 1,0
3 6 3 8 4 2 8 4 2
1 3 3 1 3 1
(0,0,1,0,0,0,1,2) (1) 0,0,1, , , 0,0, = 0,0,0, , , 0,1,
8 4 2 8 4 2

These would be used for the 3rd tableau which is given below
Step # 3
Basic z x1 x2 s1 s2 s3 s4 Solution
z 1 0 0 0 0 21
x1 0 1 1 - 0 0 3
x2 0 0 1 -1/8 0 0 3/2
s3 0 0 0 3/8 -5/4 1 0 5/2
s4 0 0 0 1/8 -3/4 0 1

Hence the answer is

3 5 1
Maximum value is 21, 1 = 3, 2 = , 1 = 0, 2 = 0, 3 = , 4 =
2 2 2

14.6.1 Special cases of Linear Programming

1) Degeneracy: An LP is degenerate if in a basic feasible solution, one of the basic variables takes
on a zero value.
2) Alternative optimal: Alternative optimal means that the objective value remains the saqme but
the decision variables input values ar different.
3) Unbounded solution: In case increasing/decreasing the decision variables keep
increasing/decreasing the objective function then we have an unbounded solution.
4) Non-existing or infeasible solution: In case when we do not have any solution in the feasible
space which is the objective function, then we have a non-existing or infeasible solution.

Page 22 of 31
Example # 14.7
Ankita Hemant Sahare Industries, a nationally known manufacturer of clothing, produces four
varieties of ties. One is an expensive, all-silk tie, one is an all-polyester tie, and two are blends of
polyester and cotton. The following table illustrates the cost and availability (per monthly production
planning period) of the three materials used in the production process:
Material Cost/meter (XYZ units) Material Available/Monthly (meters)
Silk 21 800
Polyester 6 3,000
Cotton 9 1,600
The firm has some fixed contracts with several major department store chains to supply ties. The
contracts require that Ankita Hemant Sahare Industries supply a minimum quality of each tie but
allow for a larger demand if Ankita Hemant Sahare Industries chooses to meet that demand, subject to
a condition that it should not exceed the maximum requirement. The following table summarizes the
contract demand for each of the four styles of ties, the selling price per tie, and the fabric requirements
of each variety.
Variety of Tie S.P. per tie Monthly Monthly Material Type of
(XYZ units) contract contract requirement Material
(Minimum) (Maximum) per tie requirement
All Silk 6.70 6,000 7,000 0.125 100% silk
All polyester 3.55 10,000 14,000 0.080 100% ploy
Poly cotton 4.31 13,000 16,000 0.100 50% poly and
Blend 1 50% cotton
Poly cotton 4.81 6,000 8,500 0.100 30% poly and
Blend 2 70% cotton
Formulate the optimization problem for Ankita Hemant Sahare Industries.

Let us define the following

x1= number of all silk ties produced per month
x2= number of polyester ties produced per month
x3= number of blend 1 poly-cotton ties produced per month
x4= number of blend 2 poly-cotton ties produced per month
First the firm must establish the profit per tie
For all silk ties (x1), each requires 0.125 yard of silk, at a cost of 21/yard. Therefore, the cost
per tie is 2.62. The selling price per silk tie is 6.70, leaving a net profit of (6.70-2.62)=4.08 per
unit of x1.

Page 23 of 31
 For all polyester ties (x2), each requires 0.080 yard of polyester, at a cost of 6/yard. Therefore,
the cost per tie is 0.48. The selling price per polyester tie is 3.55, leaving a net profit of (3.55-
0.48)=3.07 per unit of x2.
For all blend 1 poly cotton ties (x3), each requires 0.050 yard of polyester, at a cost of 6/yard
and 0.050 yard of cotton, at a cost of 9/yard. Therefore, the cost per tie is 0.75. The selling price
per polyester tie is 4.31, leaving a net profit of (4.31-0.75)=3.56 per unit of x3.
For all blend 2 poly cotton ties (x4), each requires 0.030 yard of polyester, at a cost of 6/yard
and 0.070 yard of cotton, at a cost of 9/yard. Therefore, the cost per tie is 0.81. The selling price
per polyester tie is 4.81, leaving a net profit of (4.81-0.81)=4.00 per unit of x4.

Hence the optimization problem is as follows

: 4.081 + 3.072 + 3.563 + 4.004
. . : 0.1251 800
0.082 + 0.053 + 0.034 3000
0.053 + 0.074 1600
6000 1 7000
10000 2 14000
13000 3 16000
6000 4 8500

Example # 14.8
A soap manufacturing company produces a particular brand of soap from four (4) chemicals, namely
A1, A2, A3 and A4. The three (3) ingredients in the soap are B1, B2 and B3. On any one day the
company must produce 1000 kgs of that particular brand of soap and it is know that by weight (i) at
least 10% of the soap must be B1, (ii) at least 5% of the soap must be B2 and (iii) at least 4% of the
soap must be B3. The cost structure for manufacturing per kg of the soap as well as the amount of
each ingredients is as follows

Chemical Cost (INR per kg) B1 B2 B3

A1 80 0.05 0.03 0.15
A2 100 0.10 0.07 0.02
A3 125 0.09 0.08 0.12
A4 150 0.15 0.11 0.09

If is it necessary that at least 125 kgs of chemical A3 be used, then formulate the problem as a LP
problem, where the aim is to find the cheapest way of producing the soap in a single day? Does the

Page 24 of 31
problem change if we have to find the cheapest way of manufacturing the soap in a month, if in a
month the number of working days in the shop floor is 30? For solving part two of the problem do
you need to make some assumptions, then, state them clearly.

Example # 14.9
Koshagar bank is planning to invest its total assets and deposits (that its customers have deposited
with the bank) which amounts to INR 50 crores in different investments (such as personal loans,
vehicle loans, house loans, etc.) as well adhere to the Reserve Bank of Indian (RBI) rules/regulations
and policies, as applicable. The table provides the type of loan it is planning to invest in or has to
invest as per RBI regulations, and also the percentage the bank has to disburse in order to meet the
criteria

Type of loan/deposits Interest rate Bad debt ratio

Personal 0.10 0.20
Automobile 0.15 0.12
CRR (cash reserve ratio) -0.05 0.00
Home 0.20 0.05
Commercial/farm 0.05 0.15

One should remember that bad debts cannot be recovered and earn no interest rates. There is
competition in the commercial/farm sector from other banks, hence the bank decides that it has to
invest at least 30% of its total assets plus deposits in commercial/farm loans. Home loans are the new
thrust area for the bank, hence Koshagar has decided that home loans must be 45% of all the
combined loans disbursed to personal, automobile and home loans combined together. As per the
policy of the bank the difference in the total interest in personal and CRR should be more than INR
5.6 crores. Also the total amount of bad debts cannot exceed 10% of the loan disbursed. Model this
problem so that the bank is able to invest such amounts in the different categories keeping in mind
that it maximizes its profit earned from the interest rates.

14.6.2 Dual Problem

The dual of a LP (called the primal), is also the complementary problem of the LP. Now in case of
maximization of LP the dual is minimization, while for minimization in LP the dual is the
maximization and reaching the optimal solution of LP also ensures the optimal solution of dual or
vice versa.

For minimization of primal or so as to say the maximization of dual

Primal

Page 25 of 31
Optimal solution

Dual

For maximization of primal or so as to say the minimization of dual

Dual

Optimal solution

Primal

14.6.3 General Primal and Dual Problem

1) A dual variable is defined for each primal constraint equation
2) A dual constraint is defined for each primal variable
3) The constraint coefficients of primal variable defined the LHS coefficients of the dual constraints
and its objective coefficients define the RHS
4) The objective coefficients of the dual equal the RHS of the primal constraints equations

Example # 14.10
Primal Problem Formulation
Maximize 5x1 + 12x2 + 4x3

s.t:
x1 + 2x2 + x3 10
2x1 - x2 + 3x3 = 8
x1, x2, x3 10

Hence we have
Maximize 5x1 + 12x2 + 4x3 + 0x4

s.t:
x1 + 2x2 + x3 + x4 = 10
2x1 - x2 + 3x3 + 0x4 = 8
x1, x2, x3, x4 10

Page 26 of 31
Dual Problem Formulation
Minimize 10y1 + 8y2

s.t:
y1 + 2y2 5
2y1 - y2 12
y1 + 3y2 4
y1 + 0y2 0
y1, y2 are unrestricted

Example # 14.11
Write the dual of Example 14.5

Example # 14.12
Write the dual of Example 14.6

14.6.4 General Primal and Dual Problem and the concepts of Economics
General primal problem General dual problem
n

c x
m
Maximize
j =1
j j Minimize b y
i =1
i i

s.t : s.t :
n m

a x
j =1
ij j bi i = 1,2,, m a ij yi c j j = 1,2,, n
i =1

xj 0 j = 1,2,, n yi 0 i = 1,2,, m

Thus if the primal is the resource allocation of ( = 1,2, . , ) number of resources for ( =
1,2, . , ) number of activities, such that we are interested to find the maximum, then the dual is the reverse
of the primal. Here is the maximum availability of resource.

Thus the dual variable , denotes the worth per unit of resource . Hence the shadow price of
resource replaces the name worth per unit and it signifies the worth of addition or subtracting the
resource per unit. Now in case of the equality constraint (for the constraint) being true in the
primal, would imply that the resource utilization has been done at its best level hence is zero, which
would mean that there are no unused resources for the ith resource. This concept of shadow prices,
dual solution has a huge amount of implication for LP as well as for the case when we know we have
DRTS, CRTS or IRTS and the general solution strategy which we have discussed so far is for the case

Page 27 of 31
of CRTS. Similar concept of dual problems and the concept of shadow prices has also been proved for
the case of NLP.

14.6.5 Other Solutions Techniques

1) Big M method
2) NW corner method
3) Hungarian method

For network flows we have different algorithm to solve the max flow and min cut problems, but we
will not discuss those due to paucity of time

14.7 Integer Programming

An Integer Programming (IP) is a mathematical formulation in which all the variables, =
(1 , 2 , . , ), are restricted to be integers and not continuous. In case some of the variables are
continuous, while the rest are integer, then this class of problem is termed as Mixed Integer
Programming (MIP). Both IP or MIP can be either of LP or NLP flavour, and in case of the former,
i.e., LP, the problems are known as Integer Linear Programming (ILP) or Mixed Integer Linear
Programming (MILP). While on the other hand for the latter, i.e., NLP, the problems are Non-Linear
Integer Programming (NLIP) and Mixed Non-Linear Integer Programming (MNLIP). In general if
one has to write the general model for these classes of problems the differences are with respect to
how the variables and/or the objective and constraints are written. Obviously depending on what type
of problem we have the formulation is different and obviously the solution methodology is also
specific to the fact of what class of problem we have. In all these formulations it is seen that there are
few unique solutions methodologies or steps, if one may say so, which are utilized irrespective of the
fact that we have ILP, MILP, NLIP, MNLIP, etc., and those which are of significance and which
demands a good attention from our perspective are (i) Gomory cut method; (ii) Bender Algorithm,
(iii) Cutting plane Algorithm; (iv) Branch and Bound Method

14.8 Non-Linear Programming

In interger linear programming problem (ILP) we have the following form of problem formulation

Page 28 of 31
 n
Maximize c x
j =1
j j

s.t :
n

a x
j =1
ij j bi i = 1,2,, m

x j 1,2, j = 1,2,, n

14.9 Integer Linear Programming Algorithm

Step #1: Relax the solution space of the ILP by assuming the variables are not restricted to be integers.

Step # 2: Solve the LP and identify the continuous optimum.

Step # 3: Starting from the continuous optimum point ass the constraints that iterative modify the LP ti ILP
and continue doing this step till all the integer variables have been accounted for.

14.10 Branch and Bound Algorithm

This is one of the most powerful algorithm for solving ILP and other classes of problems, and very simply
stated the steps are as follows

Step # 1: Obtain an optimal solution to the given LP ignoring the restriction of integers.

Step # 2: Test the integrability of the optimum solution obtained in step # 1 and there can be two cases of the
same.
(a) If the solutions are integers, then the current solution is the optimal one
(b) If the solutions are not all integers, then go to step # 3

Step # 3: Consider the objective function as upper bound and obtain the lower bound by rounding off to
integers the decision variables

Step # 4: Let the optimum value xj*, i.e., the decision variables, be not an integer, then sub-divide the LP into
two classes, one with LP with the constraint xj* [xj*] (sub-problem # 1) and other LP with the constraint xj*
[xj*] +1 (sub-problem # 2)

Step # 5: Solve both the sub-problems and there can be three cases.
Case # 1: The optimum solutions of both are integers, then choose the one which is more (in case of
maximization)

Page 29 of 31
 Case # 2: Optimum solution of one is integer and other is infeasible, then choose the one which is
feasible and integer
Case # 3: Optimum solution of one is integer and other is feasible but non-integer, then repeat steps # 3
to # 4 for the non-integer problem

Step # 6: Repeat steps # 3 to # 5 until all integer values are obtain

Step # 7: Choose the one solution for which the optimum value of the objective function is as desired
(maximum for maximization problem while minimum for minimization problem)

We will see one problem solution from the finance perspective and give its solutions only.

14.11 Lagrangian Method (For Solving NLP when the constraints are equality constraints)
Consider you have the following n dimensional problem for xi, i = 1,2, n , such that we have

Maximize f ( x1 , x2 ,, xn )
s.t:
g j ( x1 , x2 ,, xn ) = c j j = 1,2, m

Now consider the Lagrangian multipliers, j, ( j = 1,2, m ) , such that we have the following
Lagrangian given as

L( x1 , x2 ,, xn , 1 , 2 ,, m ) = f ( x1 , x2 ,, xn ) j {g j ( x1 , x2 ,, xn ) c j }
m

j =1

Now if we pay a close attention to the above equation then it becomes obvious that what we are
( )
interested in finding is the set x1* , x2* ,, xn* , 1* , *2 ,, *m which will maximize f ( x1 , x2 ,, xn ) , i.e.,

( )
f x1* , x2* ,, xn* , say f * , such that the following will hold, i.e.,

L
(x1 , x2 ,, xn , 1 , 2 ,, m ) = 0 and i = 1,2, n
xi
L
(x1 , x2 ,, xn , 1 , 2 ,, m ) = 0 j = 1,2, m
j

Page 30 of 31
This is to say we are now solve an unconstrained maximization problem, where the objective function
is L( x1 , x2 ,, xn , 1 , 2 ,, m )

14.12 Some Other Optimization Problems (only formulation)

14.12.1 Mixed Integer Linear Programming (MILP)

n
Maximize c x
j =1
j j

s.t :
n

a x
j =1
ij j bi i = 1,2,, m

x j1 1,2, j1 = 1,2,, l
x j2 0 j1 = l + 1, l + 2,, n

14.12.2 Mixed Integer Non-Linear Programming (MINLP)

n
Maximize c x
j =1
j
2
j

s.t :
n

a x
j =1
ij j bi i = 1,2,, m

x j1 1,2, j1 = 1,2,, l
x j2 0 j1 = l + 1, l + 2,, n

Page 31 of 31