Académique Documents
Professionnel Documents
Culture Documents
AND FINANCE
School of Economics and
Business Administration
Academic year 2015
Contents
1 Linear Algebra
1.1 Matrix Algebra (Chapter 8 of Simon and Blume) . . . . . . . . . . . . . .
1.1.1 Operations with matrices . . . . . . . . . . . . . . . . . . . . . . .
1.1.2 Laws of the matrix Algebra . . . . . . . . . . . . . . . . . . . . . .
1.1.3 Trasposition of a matrix . . . . . . . . . . . . . . . . . . . . . . . .
1.1.4 Special kinds of matrices . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Determinant of a matrix (Chapter 9 of Simon and Blume) . . . . . . . . .
1.2.1 Properties of the determinant . . . . . . . . . . . . . . . . . . . . .
1.3 Rank of a matrix (rk(A)) . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3.1 Using determinants . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3.2 Using Gauss-Jordan method . . . . . . . . . . . . . . . . . . . . . .
1.3.3 Using vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4 Invertible matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4.1 Inverse matrix using cofactors . . . . . . . . . . . . . . . . . . . . .
1.4.2 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.5 Systems of linear equations . . . . . . . . . . . . . . . . . . . . . . . . . .
1.5.1 Rouche-Frobenius theorem . . . . . . . . . . . . . . . . . . . . . .
1.5.2 Solution of systems of linear equations . . . . . . . . . . . . . . . .
1.6 Eigenvalues and eigenvectors of a matrix . . . . . . . . . . . . . . . . . . .
1.6.1 Diagonalization of a matrix . . . . . . . . . . . . . . . . . . . . . .
1.6.2 Properties of diagonalization . . . . . . . . . . . . . . . . . . . . .
1.7 Application: linear difference equations (Chapter 23 of Simon and Blume)
1.7.1 One dimensional equations . . . . . . . . . . . . . . . . . . . . . .
1.7.2 k-dimensional systems . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
1
1
3
4
4
5
6
8
8
9
9
11
11
12
13
14
14
20
22
25
25
25
25
2 Multivariate Calculus
2.1 Functions (Chapter 13 of Simon and Blume)
2.1.1 Special functions . . . . . . . . . . . .
2.1.2 Classification of functions . . . . . . .
2.1.3 Composition of functions . . . . . . .
2.2 Derivatives of multivariate functions . . . . .
2.2.1 Partial Derivatives . . . . . . . . . . .
2.2.2 The total differential . . . . . . . . . .
2.2.3 The Chain Rule . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
27
27
28
30
30
31
31
32
33
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
CONTENTS
.
.
.
.
.
.
.
.
.
34
34
35
35
36
37
38
39
39
3 Optimization
3.1 Unconstrained optimization . . . . . . . . . . . . . .
3.1.1 Theorem 1 . . . . . . . . . . . . . . . . . . .
3.1.2 Theorem 2 . . . . . . . . . . . . . . . . . . .
3.1.3 Theorem 3 . . . . . . . . . . . . . . . . . . .
3.2 Optimization with equality constraints . . . . . . . .
3.2.1 Two variables and one equality constraint . .
3.2.2 Several equality constraints . . . . . . . . . .
3.3 Optimization with inequality constraints . . . . . . .
3.3.1 One inequality constraint . . . . . . . . . . .
3.3.2 Several inequality constraints . . . . . . . . .
3.4 Kuhn-Tucker formulation . . . . . . . . . . . . . . .
3.4.1 Optimization with mixed constraints . . . . .
3.4.2 Envelope theorem . . . . . . . . . . . . . . .
3.4.3 Special functions . . . . . . . . . . . . . . . .
3.4.4 Concave and quasiconcave functions (Chapter
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
21 of Simon and Blume)
43
43
43
44
44
45
45
51
54
54
59
64
66
67
68
70
4 Analysis
4.1 Sequences of real numbers . . . . . . .
4.1.1 Convergent sequence . . . . . .
4.2 Open sets . . . . . . . . . . . . . . . .
4.3 Continuity of functions . . . . . . . . .
4.3.1 Continuous function at x0 . . .
4.3.2 Uniformly continuous function
.
.
.
.
.
.
73
73
73
75
75
75
76
2.3
2.4
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Chapter 1
Linear Algebra
1.1
A matrix is simply a rectangular array of numbers. So, any table of data is a matrix.
A= .
..
..
..
..
.
.
.
am1 am2 amn
The dimension of the matrix is dim(A) = m n, being m the number of rows and n
the number of columns.
aij : elements of the matrix in i th row and j th column.
Mmn = {Set of matrices of dim = m n}
1.1.1
Addition of matrices
Being A = (aij ) and B = (bij ) two matrices such that dim(A) = dim(B),
matrix of A and B is defined as:
b11 b12
a11 a12 a1n
a21 a22 a2n b21 b22
A + B = (aij + bij ) = .
..
..
.. + ..
..
..
..
.
.
. .
.
.
bm1 bm2
am1 am2 amn
1
the addition
b1n
b2n
..
.
bmn
1. Linear Algebra
a11 + b11
a21 + b21
..
.
a12 + b12
a22 + b22
..
.
..
.
a1n + b1n
a2n + b2n
..
.
am1 + bm1
am2 + bm2
amn + bmn
Example.
1 2
0 7
1 5
4
0 4 2
1
6 2
8 + 5 4 0 = 5 11 8
5
1 0 2
2 5 3
Scalar multiplication
Given A Mmn and a real number R, A is the matrix (aij ):
A =
a11
a21
..
.
a12
a22
..
.
..
.
a1n
a2n
..
.
am1
am2
amn
a11
a21
..
.
a12
a22
..
.
..
.
a1n
a2n
..
.
am1
am2
amn
Example.
1 2 4
5 10 20
5 0 7 8 = 0 35 40
1 5 5
5 25 25
Matrix multiplication
Definition 1: Given a row matrix A and a column matrix B
A = (a1 , a2 , . . . , an )
B=
b1
b2
..
.
bn
1. Linear Algebra
we define the multiplication of A B as the scalar:
b1
n
b2
X
A B = (a1 , a2 , . . . , an ) . = a1 b1 + . . . + an bn =
ai b i
..
i=1
bn
n
X
aik bkj
k=1
1 2
0 7
1 5
4
0 4
8 5 4
5
1 0
2
6 12 6
0 = 43 28 16
2
20 16 12
1.1.2
Associative laws.
A, B, C Mmn :
(A + B) + C = A + (B + C)
1. Linear Algebra
A Mnm and B, C Mmr :
A (B + C) = A B + A C
A, B Mnm and C Mmr :
(B + C) A = A C + B C
1.1.3
Trasposition of a matrix
A=
a11
a21
..
.
a12
a22
..
.
..
.
a1n
a2n
..
.
am1
am2
amn
At =
a11
a12
..
.
a21
a22
..
.
..
.
am1
am2
..
.
a1n
a2n
amn
Example.
1
At =
0
3
1 1 0 3
7
0 7
A= 0
1 5 5 9
1.1.4
if i 6= j
Example.
1
0
0
0
0
7
0
0 15
0 1
7 5
0 5
7 9
1. Linear Algebra
Upper (lower) triangular matrix. A = (aij ) such that:
aij = 0
if i > j( if i < j)
Example.
1 2
0 7
0 0
4
8
5
1 0
0 7
1 5
0
0
5
1 1 3
0
A = 1 7
3
0 5
Skew symmetric matrix. A Mmn such that A = At . aji = aij i 6= j and,
aii = 0 i.
Example.
0 1 3
0 2
A= 1
3 2
0
Identity matrix. Inn = In :
I=
1.2
1
0
..
.
0
0
1
..
..
.
.
0
0
0
..
.
1
a
c
b
= ad bc
d
1. Linear Algebra
3 3:
a11 a12
a21 a22
a31 a32
a13
a23
a33
Examples.
0 4
5 4
1 0
1.2.1
2
0
2
= a11 a22 a33 +a12 a23 a31 +a13 a21 a32 [(a31 a22 a13 )+(a32 a23 a11 )+(a33 a12 a21 )]
3 2
6 4 = 0
= 0 4 2 + 4 0 (1) + 2 (5) 0 2 4 (1) 0 0 0 2 (5) 4 = 48
1. |A| = |At |
2. |A B| = |A||B|
A, B Mnn
A, B Mnn
3 0 0
1 2 0
3 2 6
2 1 2
0
0
0
4
= 3 (2) 6 4
1. Linear Algebra
10. If a linear combination of several rows (columns) is added to
|A| does not change.
Example.
3 4 R2 =3R2 2R1 3
4
6=
0 14
2 2
3 4 R2 =3R2 2R1
3
4
(1/3)
2 2
0 14
3
2
4 R2 =R2 2/3R1
3
0
4
14/3
0 4 2
A = 5 4 0
1 0 2
4
M11 =
0
0
=8
2
Example.
i-th row
j-th column
1. Linear Algebra
3 2
1 1
2 1
1 0 3 2
2+1
= 1 (1)
2 0
3 2
0 5
1 2
2 1 2 4
1
5
4
3
+ 0 (1)2+2 3
2
1
0
2
3 2 1
3 2 1
+(3) (1)2+3 3 2 5 + 2 (1)2+4 3 2 0 =
2 1 2
2 1 4
1
5
4
1 (29) + 0 + 3 28 + 2(7) = 99
1.3
1.3.1
3
1
3
2
2
1 1 C3 = C3 + 3C1 3 2 10 5
0
0
0 3 2 C4 = C4 2C1 1 0
3 2
9
1
2
0 5
2 1 8
0
1 2 4
2 10 5
9 1 = 1 (99) = 99
= 1 (1)2+1 2
1 8
0
The rank of a matrix A Mmn is the order of the greatest minor of A distinct to zero.
It is said the rank of a matrix A is p and it is written rk(A) = p if:
(i) A minor of order p exits and is nonzero.
(ii) All the minors of order greater than p do not exist or are zero.
1 4 2
Example. Given the matrix A = 5 2 1 , we have that:
1 0 2
(a) Any element of the matrix is a minor of order 1 of the matrix. As at least one element
of the matrix is nonzero, (for instance, a11 = 1), then, at least the rank of the matrix
is : rk(A) 1. a1 1 is the principle minor of order 1.
1. Linear Algebra
1 4
(b)
5 2
4 2
0 2
nonzero,
(principle minor of order 2), 1 4 , 1 2 , 1 2 , 4 2 ,
1 0 5 1 1 2 2 1
are all the minors of order 2 of the matrix A. As at least one of them is
1.3.2
The Gauss-Jordan method is used to reduce a matrix to row echelon form. The rank of the
matrix is the number of nonzero rows in its row echelon form.
To reduce a matrix into its rwo echelon form, we need to apply elementary row operations:
Interchange two rows of a matrix.
Change a row by adding it a linear combination of another row.
A row of a matrix A has k leading zeros if the first k elements of the row are all zeros
and the (k + 1)-th element of the row is not zero.
A matrix is in row echelon form if each row has more leading zeros than the row
preceding it (unless the row contains only zeros, in which case the subsequent rows must
contain only zeros.
Example.
1 0
A= 2 3
4 6
1.3.3
R2 = R2 2R1
3
1
R3 = R3 4R1
0
6
11
0
0 3
1
R3 =R3 2R2
0
3 0
6 1
0
0 3
3 0
0 1
Using vectors
10
1. Linear Algebra
3. (
u + v) + w
=u
+ (
v + w)
u +
vS
Examples.
1. {0} is a subpace of Rn .
2. S = {(1, 0), (0, 0), (0, 1)} is a subset of R2 , not a subspace, as:
(1, 0) + (1, 0)
/S
3. T = {(x, y) R2 |x y = 0} is a subspace of R2
Needed concepts related to Rn : vectors, addition of vectors and scalar multiplication.
Vectors u
1 , u
2 , . . . , u
n in Rn are linearly independent if and only if (Chapter 11 Simon
and Blume):
1 u
1 + 2 u
2 + . . . + n u
n =
n
X
i u
i = 0
i=1
with i = 0, . . . , n
For scalars i ,
1 u
1 + 2 u
2 + . . . + n u
n
11
1. Linear Algebra
6= 0
Then rk(A) = 2 =number of linearly independent vectors. Then both vectors are
basis of R2 .
In the vector space R3 , D = {(1, 1, 0), (1, 2, 1), (3, 0, 5)} is a basis R3 , as it is
formed by 3 linearly independent vectors.
1.4
Invertible matrices
Given A Mnn , its inverse matrix, A1 , its the matrix that accomplishes:
A1 A = AA1 = In
(A has to be a non singular matrix, |A| 6= 0).
1.4.1
A1 =
1
(Ad )t
|A|
|A|
C11
C12
..
.
C21
C22
..
.
..
.
Cn1
Cn2
..
.
C1n
C2n
Cnn
12
1. Linear Algebra
with Cij the cofactor of aij .
Example.
2 4 7
Ad = 0 2 2
2 4
6
2
A= 2
3
A1
1.4.2
2 2
1 0
2 2
|A| = 2
1 0
= 2 1
7/2 1
2
0 2
(Ad )t = 4 2 4
7 2 6
1
2
3
Properties
(At )1 = (A1 )t
|A1 | =
If A is nonsingular:
1
|A|
Am = A A A . . . A
(m times)
Am = (A1 )m
Am is invertible and (Am )1 = (A1 )m For any scalar 6= 0, A is invertible and:
(A)1 =
1 1
A
13
1. Linear Algebra
1.5
=
=
..
.
b1
b2
..
.
= bm
a11
a21
..
.
a12
a22
..
.
..
.
a1n
a2n
..
.
am1
am2
amn
x1
x2
..
.
xn
b1
b2
..
.
bm
Therefore:
AX = B
(A|B) =
a11
a21
..
.
a12
a22
..
.
..
.
a1n
a2n
..
.
am1
am2
amn
|
|
|
|
b1
b2
..
.
bm
14
1. Linear Algebra
1.5.1
Rouch
e-Frobenius theorem
A system of linear equations with m equations and n variables has a solution if and only if:
rk(A) = rk(A|B)
Then:
If rk(A) = rk(A|B), the system has solution.
(a) If rk(A) = rk(A|B) = n (n: number of variables), the system has a unique
solution.
(b) If rk(A) = rk(A|B) < n the system has infinite solutions.
Otherwisw if rk(A) 6= rk(A|B), the system has no solution.
1.5.2
=
=
=
1
2
0
1yz
. And plug this
2
1yz
2
= 3
= 1
=
3z
. We plug this result into equation three and solve
3
15
1. Linear Algebra
x
y
z
1yz
2
3z
=
3
= 3/4
Elimination of variables
Use elementary equation operations:
Add a multiple of one equation to another.
Multiply both sides of one eq. by a non-zero scalar.
Interchange two equations.
Fact: If one system of linear equations is derived from another elementary equation
operationed, then both systems have the same solutions (systems are equivalent).
Example.
2x + y + z
x + 2y + z
x + y + 2z
=
=
=
1
2
0
2
1
1
1 1
x
1
2 1 y = 2
1 2
z
0
2 1 1
x
1
0 3 1 y = 3
0 1 1
z
2
Another equivalent system R3 = 3R3 + R2 :
2 1 1
x
1
0 3 1 y = 3
0 0 4
z
3
16
1. Linear Algebra
Matrix methods
1. For systems with a unique solution. If in a system of linear equations, rk(A) =
rk(A|B) = n (n linearly independent equations), the unique solution can be found
using:
(a) Cramers Rule. The unique solution of the system X = (x1 , x2 , . . . , xn ) of the
system AX = B is:
xi =
Bi
|A|
i = 1, . . . , n
where Bi is the matrix A with the independent column matrix B replacing the
i th column of A:
xi =
a11
a21
..
.
an1
..
.
b1
b2
..
.
..
.
a1n
a2n
..
.
bn
|A|
ann
Example.
3 2
|A| = 2 1
1 1
1 2
1 1
2 1
x1 =
8
3x1 + 2x2 + x3 = 1
2x1 + x2 2x3 = 1
x1 + x2 x3
= 2
3 2 1
x1
1
2 1 2 x2 = 1
1 1 1
x3
2
1
2 = 8 6= 0, I can use the Cramers rule:
1
3
1
1
1
2 1 2
2
1 2 1
1
= 1
x2 =
=1
8
3 2 1
2 1 1
1 1 2
x3 =
=2
8
(b) Inverse matrix method. If rk(A) = n, then A is a non singular matrix, |A| 6= 0.
Then A1 exists and:
A1 AX = A1 B
In X = A1 B
17
1. Linear Algebra
X = A1 B
Example.
3x1 + 2x2 + x3 = 1
2x1 + x2 2x3 = 1
x1 + x2 x3
= 2
3 2 1
x1
1
2 1 2 x2 = 1
1 1 1
x3
2
3 2 1
|A| = 2 1 2 = 8 6= 0, then A1 exist and the solution of the system
1 1 1
will be X = A1 B:
1
3 5
4 4 4
2. For systems with one or infinite solutions. In any system of linear equations,
row echelon form is applied to the augmented matrix of the system, (A|B).
Straightforward, rank(A) and rank(A|B) can be obtained, and the number of solutions
of the system (0, 1 or ) can be determined.
(a) If the system has a unique solution (rk(A) = rk(A|B) = n). The substitution
method, or Cramer or the inverse method is applied to obtain the unique solution
over the equivalent system with the row echelon form of the augmented matrix,
(A|B)r .
Example.
3x1 + 2x2 + x3
2x1 + x2 2x3
x1 + x2 x3
= 1
= 1
= 2
18
1. Linear Algebra
3 2 1
x1
1
2 1 2 x2 = 1
1 1 1
x3
2
The augmented matrix is:
3 2
(A|B) = 2 1
1 1
1 | 1
2 | 1
1 | 2
3 2
(A|B) = 2 1
1 1
R2 + 2R1
1
R3 3R1
0
1 | 1
1 1
R1 R3
2 | 1 3 2
1 | 2
2 1
1 1 | 2
1
3R3 +R2
3 4 | 5 0
1 4 | 7
0
1 | 2
1 | 1
2 | 1
1 1
3 4
0 8
| 2
| 5
| 16
1
The solution of the system is: X = 1
2
(b) If the system has infinite solutions (rk(A) = rk(A|B) < n). From the set of n
variables, n rk(A) variables are taken as parameters. Thus, a subsystem of k
equations and k variables is obtained.
The rest of the variables xi are calculated in terms of these n rk(A) variables.
Example.
x1 x2 + x3
4x1 + 5x2 5x3
2x1 + x2 x3
x1 + 2x2 2x3
1
4
2
1
=
=
=
=
1
4
2
1
1 1
1
x1
4
5 5
x2 =
2
1 1
x3
2 2
1
19
1. Linear Algebra
The augmented matrix of the system:
1
4
(A|B) =
2
1
1 1 | 1
5 5 | 4
1 1 | 2
2 2 | 1
(A|B) =
2 1 1 | 2
1 2 2 | 1
1
0
0
0
1 1 | 1
9 9 | 0
3 3 | 0
3 3 | 0
= 1
=
1
The solution of this system is X =
=
=
=
0
0
0
20
1. Linear Algebra
3 2
2 0
1 2
4
x1
0
2 x2 = 0
0
x3
0
3 2 4
As |A| = 2 0 2 = 0, then rk(A) = 2 < 3, then the systems has infinite
1 2 0
solutions and we need n r = 3 2 = 1 parameter. Making x2 = , the solution
of the system is: x1 = 2, x3 = 2, R
1.6
2 1 0
Example. A = 0 0 1 .
0 0 3
The characteristic polynomial is:
2r
p() = |A rI| = 0
0
1
0
r
1
0 3r
= r(2 r)(3 r)
21
1. Linear Algebra
Example.
2
A= 0
0
1 0
0 1
0 3
Eigenspace associated to r1 = 0:
x R3 |(A 0I)
x = 0}
Er1 = {
2 1 0
x
0
0 0 1 y = 0
0 0 3
z
0
rk(A) = 2 < 3 =number of variables. Then the system has infinite solutions and
we need 3 2 = 1 parameter. The solution of the system is: x = , y = 2 y
z = 0, R. A basis of this subspace will be:
BEr1 = {(1, 2, 0)}
Eigenspace associated to r2 = 2:
x R3 |(A 2I)
x = 0}
Er2 = {
0 1 0
x
0
0 2 1 y = 0
0 0 1
z
0
rk(A 2I) = 2 < 3 =number of variables. Then the system has infinite solutions
and we need 3 2 = 1 parameter. The solution of the system is: x = , y = 0 y
z = 0, R. A basis of this subspace will be:
BEr2 = {(1, 0, 0)}
Eigenspace associated to r3 = 3:
x R3 |(A 3I)
x = 0}
Er3 = {
1 1 0
x
0
0 3 1 y = 0
0
0 0
z
0
rk(A 3I) = 2 < 3 =number of variables. Then the system has infinite solutions
and we need 3 2 = 1 parameter. The solution of the system is: x = , y =
y z = 3, R. A basis of this subspace will be:
BEr3 = {(1, 1, 3)}
22
1. Linear Algebra
1.6.1
Diagonalization of a matrix
with A, D, P Mnn
D=
r1
0
..
.
0
r2
..
.
..
.
0
0
..
.
rn
|
P = v1
|
|
v2
|
|
vn
|
2 1 0
Example. A = 0 0 1 . All the eigenvalues are different and we had that:
0 0 3
BEr1 =0 = {(1, 2, 0)}
BEr2 =2 = {(1, 0, 0)}
BEr3 =3 = {(1, 1, 3)}
Then, A is diagonalizable and D and P such that D = P 1 AP are:
0 0
D= 0 2
0 0
0
1 1
0 P = 2 0
3
0 0
1
1
3
i = i = dim(Eri ) = i = 1, 2, . . . , p
23
1. Linear Algebra
D=
r1
0
..
.
0
r1
..
.
0
..
.
..
.
..
.
..
.
..
.
..
.
..
.
0
0
..
.
ri
0
0
..
.
..
.
..
.
..
.
..
.
0
..
0
..
.
0
ri
..
.
0
..
.
..
.
..
.
0
0
..
.
rn
0
0
0
..
0
0
..
.
0
..
.
0
rn
The matrix P is formed by the eigenvectors in columns. The order of the eigenvectors
in matrix P has to be the one corresponding to the eigenvalues in matrix D.
1
Example. Given A = 1
1
1 1
1 1
1 1
If A is diagonalizable, i = dim(Eri ) = i i = 1, 2, . . . , p.
1r
p(r) = |A I| = 1
1
1
1r
1
1
1
1r
= (1 r)3 + 2 3(1 r) = r2 (r 3)
Er1
1
1
1
= {
x R3 |(A 0I)
x = 0}
1 1
x
0
1 1 y = 0
1 1
z
0
24
1. Linear Algebra
We can see that 1 = dim(Er1 =0 ) = 2 = 1 .
Calculating the basis of the eigenspace associated to r2 = 3:
x R3 |(A 3I)
x = 0}
Er2 = {
2 1
1
x
0
1 2 1 y = 0
1
1 2
z
0
rk(A) = 2 < 3 =number of variables. We need 3 2 = 1 parameter. The solution of
ther system is: x = , y = y z = , , R. Then, a basis of this eigenspace will
be:
BEr2 =3 = {(1, 1, 1)}
0
D= 0
0
0 0
1 1 1
0 0 P = 1
0 1
0 3
0
1 1
with A, D, P Mnn
25
1. Linear Algebra
1.6.2
Properties of diagonalization
If A n n is diagonalizable:
1. |A| = ni=1 ri , being r1 , r2 , . . . , rn its eigenvalues.
2. Eigenvalues of At = Eigenvalues of A.
3. If ri is an eigenvalue of A with algebraic multiplicity i , then:
dim(Eri ) = i i
4. P, D such that D = P 1 AP . Then An = P Dn P 1 n N.
5. A1 = P D1 P 1
1.7
1.7.1
In this equation:
yn+1 = ayn
The y variable can represent the amount of money in a savings account whose principal
is left untouched and whose interest is compounded once a year,
yn+1 = (1 + )yn
with the interest rate. With this difference equation, in which the variable at a certain
time n depends on its value at a time n + 1, we can solve the amount of money at a certain
time n:
1.7.2
y1
y2
..
.
=
=
ay0
ay1
..
.
yn
= an y 0
a2 y 0
k-dimensional systems
In general,
zn+1 = Azn
where zn+1 , zn , are k 1 vectors and A is a k k matrix. To solve these systems we
can face different situations:
26
1. Linear Algebra
1. A is diagonal. Then the solution of the system is the same as the one dimensional
case: zn = An z0 .
2. A has distinct real eigenvalues. In this case there is a nonsingular matrix P such that
D = P 1 AP :
r1 0 0
0 r2 0
D= .
..
..
..
..
.
.
.
0 0 rk
In this case, we can solve this system as the one dimensional case:
z1
z2
..
.
=
=
Az0
Az1
..
.
zn
An z0
A2 z0
Chapter 2
Multivariate Calculus
2.1
A function from a set A to a set B is a rule that assigns to each object in A one object in
B.
f :AB
The elements of set A for which the function f is defined is called the domain of the
function f ; the set B for which the function f takes its values is called the target of the
function f .
y = f (x) is said to be the image of x under f . The set of all f (x)s for x in the domain
of f is called the image of f .
Example:
If we consider the function f : R2 R, f (x, y) = x2 + y 2 , we have that the domain of
the function is R2 and the image of the function are all the real positive values.
If we consider the function g : R R, g(x) = 1/x, its domain is all the real numbers
except 0 and its image is R {0}.
But in Economy we are mostly interested on functions that usually have two or more
variables as input:
Demand function: f : R3 R
q1 = f (p1 , p2 , y) = K1 pa1 11 pa2 12 y b1
Production function: f : R2 R
q = f (x1 , x2 ) = kxb11 xb22
27
28
2. Multivariate Calculus
Production function of 2 outputs using 3 inputs: f : R3 R2
Q(p1 , p2 , y) = (q1 (p1 , p2 , y), q2 (p1 , p2 , y)) = (K1 pa1 11 pa2 12 y b1 , K2 pa1 21 pa2 22 y b2 )
2.1.1
Special functions
Linear functions (f : Rk Rm )
Functions that preserve the vector space structure:
f (x + y) = f (x) + f (y)
f (rx) = rf (x)
x, y Rk
x Rk , r R
x Rk
Quadratic forms (f : Rk R)
Q(
x) = X t AX = (x1 , x2 , . . . , xn ) .
..
..
.. ..
..
.
.
. .
xn
a1n a2n ann
Developing this form we get:
Q(
x) = X t AX = (x1 , x2 , . . . , xn )
n
X
aii x2i + 2
i=1
1
Q(x, y, z) = (x, y, z) 1
0
1
0
x
5 5/2 y
5/2
3
z
n
n
X
X
i=1 j=i+1
aij xi xj
29
2. Multivariate Calculus
xy: 12 or 21.
yz: 23 or 32.
But sometimes the calculus of the eigenvalues is not easy and we can classify the
quadratic forms by using the principle minors of
the matrix A.
a
a12
Given Q(
x) = X t AX and let |A1 | = a11 , |A2 | = 11
, . . . , |An | = |A| be the principle
a12 a22
minors of A.
1. Q is positive def inite |Ai | > 0, i = 1, . . . , n.
5. Q is indef inite if |An | = |A| 6= 0 and conditions of cases (1) and (3) are not satisfied.
6. Q is indef inite if |An | = |A| = 0 y |Ai | 6= 0 i = 1, . . . , n 1 and conditions of cases
(2) and (4) are not satisfied.
Monomials
A function f : Rk R is a monomial if it can be written as:
f (x1 , . . . , xk ) = cxa1 1 xa2 2 . . . xakk
c R, ai 0
30
2. Multivariate Calculus
Polynomial
A function f : Rk R is called a polynomial function if f is the finite sum of monomials
on Rk .
The degree of the polynomial is the highest degree among the different monomials.
Example:
f (x, y, z) = x3 + 2xyz + y 4
f is a polynomial of degree 4.
2.1.2
Classification of functions
f (x) 6= f (y) x 6= y
Examples.
g(x) = x2 is not one-to-one as for x = 1 and x = 1, we have that f (1) = 1.
f (x) = x is one-to-one as for two any different values of x, their images are always
different.
f : R2 R, f (x, y) = x2 + y 2 . f is not surjective as the image of f is R+ instead of
all R. Neither is one-to-one since f (1, 0) = f (0, 1) = 1.
2.1.3
Composition of functions
x A
Examples:
f : R R and g : R R, f (x) = sin(x), g(x) = x2 .
(g f )(x) = g(f (x)) = g(sin(x)) = sin2 (x)
31
2. Multivariate Calculus
2.2
2.2.1
Partial Derivatives
32
2. Multivariate Calculus
Q1 P1
%change in demand
P1 Q1
%change in own price
Q1 P2
I Q1
2.2.2
Intuitively, if F : R2 R:
F
(x , y )x
x
F
F (x , y + y) F (x , y )
(x , y )y
y
F (x + x, y ) F (x , y )
so we expect that:
F (x + x, y + y) F (x , y )
F
F
(x , y )x +
(x , y )y
x
y
or if F : Rk R, with x = (x1 , . . . , xk ):
F (x1 + x1 , . . . , xk + xk ) F (x1 , . . . , xk )
F
F
(x )x1 + . . . +
(x )xk
x1
xk
Then,
dF = F (x + x) F (x )
Thus,
dF =
F
F
(x )dx1 + . . . +
(x )dxk
x1
xk
DF (x ) =
F
F
(x ), . . . ,
(x )
x1
xk
33
2. Multivariate Calculus
If F : Rk Rm , F = (f1 (x1 , . . . , xk ), f2 (x1 , . . . , xk ), . . . , fm (x1 , . . . , xk )):
F (x + x) F (x )
f1
(x )
xk
..
.
fm
(x )
xk
f1
(x )
x1
..
.
fm
(x )
x1
x1
.
.
.
xk
DF (x ) =
f1
(x )
x1
..
.
fm
(x )
x1
f1
(x )
xk
..
.
fm
(x )
xk
Example.
3/2
1 2
Consider the pair of demand functions q1 = 6p2
1 p2 y and q2 = 4p1 p2 y . Calculate
dq1 =
q1
q1
q1
(x )dp1 +
(x )dp2 +
(x )dy
p1
p2
y
dq1 = 3dp1 + 1.5dp2 + 4.5dy
dq2 =
2.2.3
32
32
16
dp1 dp2 + dy
9
27
3
Let g(t) = f (x(t)). The derivative of the composite function is the derivative of the
outside function f (x) (evaluated at the inside of the function) times derivative of the
inside function:
g (t) = f (x(t)) x (t)
Let g(t) = f (x1 (t), . . . , xk (t)). Then:
g (t) =
f
f
(x(t))x1 (t) + . . . +
(x(t))xk (t)
x1
xk
34
2. Multivariate Calculus
Let u(t) = (u1 (t1 , . . . , ts ), u2 (t1 , . . . , ts ), . . . , un (t1 , . . . , ts )), u : Rs Rn , with t =
(t1 , . . . , ts ), and f : Rn R, then the composite function g : Rs R:
g = (f u)(t) = g(t1 , . . . , ts ) = f (u(t)) = f (u1 (t), . . . , un (t))
And then,
g
f
u1
f
un
=
(t)
(t) + . . . +
(t)
(t)
ti
u1
ti
un
ti
Let F : Rk Rm , and a : R Rk . Then, the composite function g = F a = F (a(t))
is a function g : R Rm , and its derivative is:
g (t) = DF (a(t)) a (t)
Let F : Rk Rm , and A : Rs Rk . Then, the composite function H = F A is a
function H : Rs Rm , and:
DH(s) = D(F A)(s) = DF (A(s)) D(A(s))
2.2.4
Let f : Rk R. For this function we could define the matrix of second order derivatives as:
D2 f (x) =
2f
x21
2f
x1 x2
..
.
2f
(x)
x1 xk
2f
x2 x1
2f
x22
2f
(x)
xk x1
2f
(x)
xk x2
..
.
2f
(x)
x2k
This matrix is called the Hessian Matrix, that is as you can see, a symmetric matrix.
The Young's Theorem shows that mixed partials of order n, are equal if they are continuously.
2.3
35
2. Multivariate Calculus
2.3.1
2.3.2
1 (k)
1
f (a)hk +
f (k+1) (a)hk+1
k!
(k + 1)!
36
2. Multivariate Calculus
In this expression, the k th order Taylor polynomial of F at x = a is:
Pk (a + h) = f (a) + f (a)h + . . . +
1 (k)
f (a)hk
k!
Defining the difference Rk (h; a) between the actual value f (a + h) and its k th order
approximation Pk (a + h) satisfies:
Rk (h; a) = f (a + h) Pk (a + h)
2.3.3
Rk (h; a)
0 as h 0.
hk
where
1 t 2
h D F (a)h + R2 (h; a)
2!
R2 (h; a)
0 as ||h|| 0 and ht = (h1 , . . . , hk ) and:
||h||2
k
1 X X 2F
1 t 2
(a)hi hj
h D F (a)h =
2!
2 i=1 j=1 xi xj
F
F
(a)h1 +
(a)h2 +
x1
x2
1 2F
2F
1 2F
(a)h21 +
(a)h1 h2 +
(a)h22 + R2 (h1 , h2 ; a)
2
2 x1
x1 x2
2 x22
Example. Compute the Taylor approximation of order two of the Cobb-Douglas
fucntion F (x, y) = x1/4 y 3/4 at (1, 1).
1
F
= x3/4 y 3/4
x
4
F
3
= x1/4 y 1/4
y
4
37
2. Multivariate Calculus
3
2F
= x1/4 y 5/4
y 2
16
3
2F
= x7/4 y 3/4
x2
16
2F
3 3/4 1/4
2F
=
=
x y
xy
yx
16
Evaluating the partial derivatives at z = (1, 1) we have that:
1
F
(z ) =
x
4
2F
3
(z ) =
x2
16
F
3
(z ) =
y
4
2F
3
(z ) =
y 2
16
2F
3
=
xy
16
Therefore,
F (1+h1 , 1+h2 ) = F (1, 1)+ 1/4 3/4
=1+
h1
h2
1
+
2
h1
h2
3/16 3/16
3/16 3/16
h1
h2
h1
3
3
3
3
+ h2 + h21 + h1 h2 h22 + R(h1 , h2 )
4
4
32
16
32
2.4
Usually the endogeneous variable is a explicit function of the exogeneous variable, that is:
y = F (x1 , . . . , xk )
but sometimes we face situations where both kinds of variables are mixed as in:
G(x1 , . . . , xk , y) = 0
Depending on G, we usually cannot solve for y, but we still want to answer the basic
question: How does a small change in one of the exogeneous variables affect the value of the
endogeneous variable?
38
2. Multivariate Calculus
2.4.1
For a given function G(x, y) = C and a specific point (x0 , y0 ), we want to answere the
questions:
(a) Does G(x, y) = C determine y as a continuous function of x near to x0 and y near y0 ?
(b) If so, how do changes in x affect y?
Theorem: let G(x, y) be a c1 function on a ball about (x0 , y0 ) in R2 . Suppose that
G(x0 , y0 ) = C and consider the expression G(x, y) = C.
G
(x0 , y0 ) 6= 0, then, there exists a c1 function y = y(x) defined on an interval I about
If
y
x0 such that:
1. G(x, y(x)) = C x I,
2. y(x0 ) = yo
G
(x0 , y0 )
3. y (x0 ) = x
G
(x0 , y0 )
y
Summarizing, we suppose that there exist a function y(x) that is solution to the equation
G(x, y) = C, G(x, y(x)) = C. Then we apply the Chain Rule to differentiate with respect
to x at x0 , and we get y (x0 ).
39
2. Multivariate Calculus
In the example, we can get the Taylors series approximation of order 1:
y1 y0 + y (x0 )x = 3 +
1
0.3 = 3.02
15
and y1 = 3.01475.
2.4.2
Let G(x1 , . . . , xk , y) be a c1 function around the point (x1 , . . . , xk , y ). Suppose that (x1 , . . . , xk , y )
satisfies:
G(x1 , . . . , xk , y ) = C
G
(x , . . . , xk , y ) 6= 0
y 1
Then, there is a c1 function y = y(x1 , . . . , xk ) defined on an open ball B about
so that:
(x1 , . . . , xk )
2.4.3
G
(x , . . . , xk , y )
y
xi 1
(x , . . . , xk ) =
G
xi 1
(x , . . . , xk , y )
y 1
=
..
.
C1
..
.
Fm (y1 , . . . , ym , x1 , . . . , xk ) = Cm
Defining y = (y1 , . . . , ym
), x = (x1 , . . . , xk ), suppose that (y , x ) is a solution of the
system above.
40
2. Multivariate Calculus
F
1
y1
.
.
.
F
m
y1
F1
ym
..
.
Fm
ym
f1 (x1 , . . . , xk )
..
.
ym
fm (x1 , . . . , xk )
=
..
.
C1
..
.
y1
xj
..
.
ym
xj
y1
..
.
f1 (x )
..
.
ym
fm (x )
yi
(y , x ), i = 1, . . . , m by evaluating at (y , x ),
xj
F
1
y
1
..
= .
Fm
y1
F1
ym
..
.
Fm
ym
1 F
1
x
j
..
.
F
m
xj
Example. Consider:
F1 (x, y, a) =
F2 (x, y, a) =
x2 + axy + y 2 1 =
x2 + y 2 a2 + 3 =
0
0
= 2, can
41
2. Multivariate Calculus
First, defining z = (x , y , a ),
x
(z )
a
..
.
y
(z )
a
F1
x
=
F2
(z )
x
2
0
2
2
1
F1
(z )
y
F2
(z )
y
0
4
2
2
F1
(z )
a
F2 =
(z )
a
So,
x = 2a
y = 2a
If a increases to 2.1, y will increase about 1.2 and x will decrease to 0.2.
Chapter 3
Optimization
3.1
Unconstrained optimization
B (x ) = {x|||x x || < }
3.1.1
Theorem 1
43
44
3. Optimization
F
(x ) = 0
xi
i = 1, . . . , k
3.1.2
Theorem 2
Definition: A set S in Rk is open if x S, there exists and open -ball about x completely
contained in S:
1. If the Hessian:
F (x ) = D F (x ) =
2F
(x )
x21
2
F
(x )
x1 x2
..
.
2F
(x )
x2 x1
2F
(x )
x22
2F
(x )
x1 xk
2F
(x )
xk x1
2F
(x )
xk x2
..
.
2F
(x )
x2k
3.1.3
Theorem 3
45
3. Optimization
3.2
3.2.1
Let f and h be c1 functions of two variables. Suppose that x = (x1 , x2 ) is a solution of the
problem:
h
(x )
x2
46
3. Optimization
h
f
(x )
(x )
x1
x1
=
f
h
(x )
(x )
x2
x2
f
f
(x )
(x )
x1
x2
=
=
h
h
(x )
(x )
x1
x2
h
f
(x )
(x ) = 0
x1
x1
h
f
(x )
(x ) = 0
x2
x2
47
3. Optimization
h(x1 , x2 ) = c
h
h
(x ) =
(x ) = 0
x1
x2
so this was the reason for the condition called the constraint qualification, which
is automatically satisfied if the constraint is linear.
Example. Find the maximum of the function f (x1 , x2 ) = x1 x2 subject to:
h(x1 , x2 )
= x1 + 4x2
= 16
48
3. Optimization
First, we have to see the constraint qualification condition, if there are critical points
of h that lie in the constraint region.
h
=1
x1
h
=4
x2
L
= x2 = 0
x1
L
= x1 4 = 0
x2
L
= (x1 + 4x2 16) = 0
and setting the derivatives equal to zero and solving for the three unknowns, we have
that x1 = 8, x2 = 2, a st = 2.
Example. Maximize the function f (x1 , x2 ) = x21 x2 subject to (x1 , x2 ) in the constraint
set C = {(x1 , x2 )|2x21 + x22 = 3}
First, we have to compute the critical points of h:
h
= 4x1
x1
h
= 2x2
x2
These derivatives are zero if (x1 , x2 ) = (0, 0), But as this point is not in the constraint
set C, the constraint qualification is satisfied and we can form the Lagrangian.
Lagrangian:
L(x1 , x2 , ) = x21 x2 (2x21 + x22 3)
Derivatives of the Lagrangian with respect to the variables and the lagrangian multipliers:
49
3. Optimization
L
= 2x1 x2 4x1 = 2x1 (x2 2) = 0
x1
L
= x21 2x2 = 0
x2
L
= 2x21 x22 + 3 = 0
Solving the system. As you can see, this is a nonlinear system of equations, so matrix
methods cannot be applied. Only Substitution and and elimination methods can be
applied.
We have the following equations:
2x1 (x2 2) = 0
x21 2x2 = 0
2
2x1 + x22 3 = 0
From the first equation we have two alternatives:
(a) 2x1 = 0 and x2 6= 0. Then x1 = 0, substituting this value in the three
equations we have that:
2x2
x22 3
=
=
0
0
From the last equation we have that x2 = 3, and then =0. Thus, we have
two possible solution points: (x1 , x2 , ) = (0, 3, 0) and (0, 3, 0).
(b) x1 6= 0 and x2 = 0. Then we have that x2 = 2 and substituting this
expression in the rest equations:
2x21
x21 42 =
+ 42 3 =
0
0
1
and x1 = 1. Thus, we have another 4 possible points:
2
(1, 1, 0.5), (1, 1, 0.5), (1, 1, 0.5), (1, 1, 0.5).
We have that =
50
3. Optimization
42 =
42 3 =
0
0
Let f and h be c1 functions. For any fixed value of the parameter c, let (x1 (c), x2 (c))
be the solution of the problem, with the corresponding multiplier (c).
Suppose that x1 , x2 and are c1 functions of c and that the constraint qualification holds
at (x1 (c), x2 (c)). Then,
(c) =
df
(x (c), x2 (c))
dc 1
So that (c) measures the rate of change of the optimal value of f with respect to c.
Example. In the previous example, we found that a maximizer of f subject to 2x21 +x22 =
3 was x1 = x2 = 1, with = 0.5.
We could redo the problem with a new constraint 2x21 + x22 = 3.3 to get x1 = x2 = 1.1
with the maximum value of f = 1.1537, an increase of 0.1537 over the original value of f .
But, by the previous theorem, we could get an approximation of this increase in f as:
51
3. Optimization
quite similar to the tru increase in f .
3.2.2
Dh(x ) =
h1
(x )
x1
..
.
hm
(x )
x1
h1
(x )
xk
..
.
hm
(x )
xk
Theorem
Let f, h1 , . . . , hm be c1 functions of k variables. Consider the previous problem and suppose
that x C is a local max or min of f on C and satisfies the NDCQ above. Then, there
exists 1 , . . . , m such that:
(x , ) = (x1 , . . . , xk , 1 , . . . , m )
is a critical point of the Lagrangian L(x, ) = f (x)1 (h1 (x)c1 ). . .m (h1 (x)cm ).
Example. Find the maximum of the function f (x, y, z) = xyz with the constraints:
h1 (x, y, z) =
h2 (x, y, z) =
x2 + y 2
x+z
= 1
= 1
52
3. Optimization
First, we check the NDCQ. The Jacobian matrix is:
h1
x
Dh(x, y, z) = h
2
x
h1
y
h2
y
h1
2x 2y
z =
h2
1
0
z
0
1
which has rank < 2 if x = y = 0. However, this point violates the first constraint, so
all the points in the constraint set satisfy the NDCQ. Forming the Lagrangian,
L
= yz 21 x 2 = 0
x
L
= xy 2 = 0
z
L
= xz 21 y = 0
y
L
=x+z1=0
2
L
= x2 + y 2 1 = 0
1
Solving the system. As you can see, this is a nonlinear system of equations, so matrix
methods cannot be applied. Only Substitution and and elimination methods can be
applied.
We have the following equations:
yz 21 x 2
xz 21 y
x2 + y 2 1
x+z1
=
=
=
=
0
0
0
0
Taking 1 and 2 in terms of x, y and z, and plugging into the first equation we have:
y 2 z x2 z xy 2
x2 + y 2 1
x+z1
=
=
=
0
0
0
53
3. Optimization
Substituting z and y 2 in terms of x and plugging into the first equation we obtain
a polynomial of third order with a root x = 1. Solving for x in the second order
polynomial and obtaining y and z, we have another 4 possible solution points:
x 0.4343
x 0.7676
y 0.9008
y 0.6409
z 0.5657
z 1.7676
Evalutaing the possible candidates in the objective function, we have that the maximizer
is x 0.7676, y 0.6409 and z 1.7676.
The meaning of the multipliers
Let f, h1 , . . . , hm be c1 functions of k variables. Let c = (c1 , . . . , cm ) be an m-tuple of
exogeneous parameters and consider the problem above. Let x1 (c), . . . , xk (c) denote the
solution with 1 (c), . . . , m (c),.
Suppose that xi , j i, j are differentiable functions of (c1 , . . . , cm ) and that NDCQ holds.
Then:
j (c1 , . . . , cm ) =
f
(x (c1 , . . . , cm ), . . . , xk (c1 , . . . , cm ))
cj 1
j = 1, . . . , m
Figure 3.3: The gradient is perpendicular to the tangent line at that point
54
3. Optimization
3.3
3.3.1
F (x ) = DF (x ) =
F
(x )
x1
F
(x )
xk
t
55
3. Optimization
F
(10000,
625)
1.5
K
F (10000, 625) = F
=
8
(10000, 625)
L
x1
x1
x1
56
3. Optimization
L
f
g
=
x2
x2
x2
L
= g(x1 , x2 ) b
and proceed as before. Note that we also require that the maximizer is not a critical
point of g.
57
3. Optimization
L
= 0, so we have to use the complementary slacknesss condition,
Thus, we cannot use
L
(x , x , ) = 0
x1 1 2
(b)
L
(x , x , ) = 0
x2 1 2
(c) [g(x1 , x2 ) b]
(d) 0
(e) g(x1 , x2 ) b
Example. Find the maximum of the function f (x, y) = xy subject to:
g(x1 , x2 ) =
x2 + y 2
g
= 2y
y
These derivatives are zero if (x, y) = (0, 0), but as these point is not on the boundary
of the region and does not satisfy x2 + y 2 = 1, the constraint qualification is satisfied
and we can form the Lagrangian.
58
3. Optimization
Lagrangian:
L(x, y, ) = xy (x2 + y 2 1)
We take the partial derivatives of the Lagrangian and the complementary slackness
condition:
L
= y 2x = 0
x
L
2.
= x 2y = 0
y
1.
3. (x2 + y 2 1) = 0
4. 0
5. x2 + y 2 1
We have the following system of equations:
y 2x = 0
x 2y = 0
(x2 + y 2 1) = 0
Solving the system. We have to analyze these systems through the lagrangian multipliers. We have in this case only one multiplier, , thus we start the analysis from the
third equation, supossing that or = 0 or x2 + y 2 1 = 0:
(a) If = 0, we have:
y
x
= 0
= 0
= 0
Thus, the solution of the system is (x, y) = (0, 0) as this point satisfies also the
inequality of the constraint x2 + y 2 1. Therefore, (0, 0) is a possible candidate
of the solution.
(b) If x2 + y 2 1 = 0, we have:
59
3. Optimization
y 2x
x 2y
x2 + y 2 1
=
=
=
0
0
0
0
x
y
= y 2 = x2
x
y
2x2 = 1
1
x =
2
1
1
Then, y = , = and we have four possible points:
2
2
1 1
1
, ,
2
2 2
1 1
1
1
1
1
1
1
1
, , ,
, , ,
, , ,
2
2 2
2
2 2
2
2 2
Notice that there are some possible candidate points with < 0.These points will be
consider if we want to calculate the minimizer of the function. Right now we want to
calculate the maximizer of the function and we consider the points with 0, and
1
1
1
1
then we find that the maximizers of this function are ( , ) and ( , ).
2
2
2
2
3.3.2
60
3. Optimization
Dg(x ) =
g1
(x )
x1
..
.
gm
(x )
x1
g1
(x )
xk
..
.
gm
(x )
xk
L
L
(x , ) = 0 . . .
(x , ) = 0
x1
xk
g1 g1 g1
x
y
z
1
1
1
g2 g2 g2
1 0
0
x
y
z
Dg(x, y, z) = g
=
g
g
3
3
3
0 1 0
0
0 1
y
z
x
g4 g4 g4
x
y
z
This matrix has rank 3, which means that at most 3 of the 4 constraints can be binding
at the same time. Besides, the NDCQ holds at any solution candidate.
The Lagrangian is:
L(x, y, z, 1 , 2 , 3 , 4 ) = xyz 1 (x + y + z 1) + 2 x + 3 y + 4 z
We take the partial derivatives of the Lagrangian and the complementary slackness
conditions:
61
3. Optimization
L
= yz 1 + 2 = 0
x
L
2.
= xz 1 + 3 = 0
y
L
3.
= xy 1 + 4 = 0
z
4. 1 (x + y + z 1) = 0
1.
5. 2 x = 0
6. 3 y = 0
7. 4 z = 0
8. i 0 i = 1, . . . , 4.
9. x + y + z 1
10. x 0
11. y 0
12. z 0
62
3. Optimization
yz 1 + 2
xz 1 + 3
xy 1 + 4
1 (x + y + z 1)
2 x
3 y
4 z
=
=
=
=
=
=
=
0
0
0
0
0
0
0
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(13)
Solving the system. We have to analyze this system through the lagrangian multipliers.
We will form a tree with different suppositions from the values of the Lagrangian
multipliers in equations 4,5,6 and 7. As we have 4 different multipliers, we can have
24 = 16 different cases.
If 1 = 0, we have that equation (13) will be:
0 = yz + 2 = xz + 3 = xy + 4
As x, y, z, 2 , 3 and 4 are positive (greater or equal than zero from the inequalities), the only possible solution is that all the terms in those 3 equations have to
be zero:
2 = 3
yz =
xz =
xy =
= 4 = 0
0 (1)
0 (2)
0 (3)
From this system, we get infinite solutions in which two of the variables are 0
and the other is different than 0. From inequality (9), we have that the value
of this variable will be in the interval [0, 1]. For all these possible candidates,
f (x, y, z) = 0. Then, all the cases from 1 to 8 in the figure have been solved.
If 1 6= 0, then from inequality (9)
1
1
1
x+y+z1
2 x
3 y
4 z
=
=
=
=
=
=
=
yz + 2
xz + 3
xy + 4
0
0
0
0
(1)
(2)
(3)
(4)
(5)
(6)
(7)
63
3. Optimization
To solve this system of equations we have to discuss it from equations (5), (6)
and (7). We analyse the different cases in the figure, from 9 to 16:
Case 9. 1 6= 0, 2 = 3 = 4 = 0 (x, y, z are 6= 0). Then the system will be:
1
1
1
x+y+z1
1 =
x+y+z1 =
yz
0
=
=
=
=
yz
xz
xy
0
= xz
(1)
(2)
(3)
(4)
=
xy
(1)
(4)
=
=
=
=
0
0
xy + 4
0
(1)
(2)
(3)
(4)
=
=
=
=
=
0
xz + 3
0
0
0
(1)
(2)
(3)
(4)
(6)
64
3. Optimization
f
(x (b), . . . , xk (b))
bj 1
j = 1, . . . , m
3.4
Kuhn-Tucker formulation
m
X
j=1
j [gj (x) bj ] +
k
X
L
g1
f
gm
=
1
...
+ vi = 0, i = 1, . . . , k
xi
xi
xi
xi
i=1
vi xi
65
3. Optimization
L
= 0, j = 1, . . . , m
j
(c) vi xi = 0, i = 1, . . . , k
(d) j 0, vi 0, i = 1, . . . , k, j = 1, . . . , m
Kuhn and Tucker worked with a Lagrangian without including the nonnegativity constraints:
e 1 , . . . , m ) = f (x)
L(x,
m
X
j01
j [gj (x) bj ]
e , v) +
L(x, , v) = L(x,
k
X
vi xi
i=1
Thus, for i = 1, . . . , k,
e
L
L
=
+ vi = 0
xi
xi
or
e
L
= vi
xi
66
3. Optimization
e
e
L
L
0, . . . ,
0
1
m
x1
1
e
e
L
L
0, . . . , xk
0
x1
xk
e
e
L
L
= 0, . . . , m
=0
1
m
3.4.1
g1
g1
(x
)
(x
)
x1
xk
..
..
gp0
gp0
(x )
(x )
x1
xk
Dgh(x ) = h
h
1
1
(x )
(x )
xk
x1
.
.
..
.
hm
hm
(x )
(x )
x1
xk
Form the Lagrangian:
L(x, , v) = f (x)
p
X
i=1
j [gj (x) bj ]
m
X
j=1
j [hj (x) cj ]
L
L
(x , ) = 0 . . .
(x , ) = 0
x1
xk
67
3. Optimization
(b) 1 [g1 (x ) b1 ] = 0, . . . , m [gm (x ) bm ] = 0
(c) hj (x ) = cj , j = 1, . . . , m
(d 1 0 . . . m 0
(e) g1 (x ) b1 , . . . , gm (x ) bm
3.4.2
Envelope theorem
Unconstrained problems
Let f (x; a) be a c1 function of x Rk and the scalar a. For each choice of a, consider the
problem:
Max f (x; a) with respect to x
Let x (a) be the solution of this problem and suppose that x (a) is a c1 function of a.
Then,
f
df
(x (a); a) =
(x (a); a)
da
a
Example. Calculate the effect of a unit increase in a on the max value of f (x; a) =
x2 + 2ax + 4a2 .
f (x) = 2x + 2a = 0 x (a) = a
68
3. Optimization
f
L
df
(x (a); a) =
(x (a); a) =
(x (a), (a); a)
da
a
a
Example. Find the maximum of the function f (x, y) = xy subject to:
g(x1 , x2 ) =
x2 + ay 2
Lagrangian:
L(x, y, ) = xy (x2 + ay 2 1)
So the optimal value of f will be decreased by 0.025 to 0.475. One can calculate
1
1
directly that the solution to the new problem is x = , y = , with maximum
2
2.2
objective value of f approx. 0.4767.
3.4.3
Special functions
(x1 , . . . , xk ), t > 0
69
3. Optimization
x1
f
f
(x) + . . . + xk
(x) = r(f (x)
x1
xk
x f (x) = rf (x)
70
3. Optimization
3.4.4
Convex set
A set U is a convex set if whenever x and y are points in U , the line segment from x to y,
is also in U .
Concave function
A real-valued function f defined on a convex set U of Rk is concave if x, y U and
t [0, 1],
f (tx + (1 t)y) tf (x) + (1 t)f (y)
Convex function
A real-valued function g defined on a convex set U of Rk is concave if x, y U and t [0, 1],
g(tx + (1 t)y) tg(x) + (1 t)f (y)
71
3. Optimization
Theorems
Theorem 1. Let f be a c1 function on an interval i in R. Then, f is concave(convex) if:
x, y I
72
3. Optimization
Theorem 3. Let f be a c2 function on an open convex subset U of Rk . Then, f is a
concave (convex) function on U if and only if the Hessian matrix D2 f (x) is negative
(positive) semidefinite x U .
Theorem 4 (Global maxima and minima). Let f be a concave (convex) function defined
on an open convex subset U of Rk . If x0 is a critical point of f , Df (x0 ) = 0, then x0
is a global maximizer (minimizer) of f on U .
Quasiconcave functions
A function f defined on a convex subset U Rk is quasiconcave if x, y U and t [0, 1],
f (tx + (1 t)y) min{f (x, f (y)}
set.
Chapter 4
Analysis
4.1
4.1.1
Convergent sequence
A sequence {xn }n = 1 converges to a limit x if > 0 there exists a number N such that,
|xn x| <
n N
lim xn = x
73
74
4. Analysis
Cauchy sequence
It is a sequence {xn }n = 1 such that > 0 there exists a number N such that,
|xn xm | <
n N, m N
Monotone sequences
A sequence is monotone increasing (decreasing) if xn1 ()xn n N. It is
monotone if it is monotone increasing or decreasing.
Bounded sequence: A sequence is bounded if there is a number B such that
|xn | B
r n
an = p 1 +
12
in which p is the investment capital, an is the accounting balance after n months, and
r is the annual compound interest rate.
This sequence is divergent, as:
r n
r n
n
= p lim 1 +
= p lim (k) =
lim p 1 +
n
n
n
12
12
75
4. Analysis
4.2
Open sets
Open ball: For a point z Rk and > 0, then the open -ball about z is:
B (z) = {x Rk |||x z|| < }
Open set: A set S in Rk is open if for each x S, there exists an open -ball
baout x completely contained in s:
x S > 0|B (z) S
Examples. (0, 1) is an open set in R, but not in R2 . S = {x Rk |||x|| 1} is
not open.
Theorem
The intersection of a finite number of open subsets of Rk is an open set.
Union of arbitrary collection of open sets is an open set.
Interior of a set S: Union of all open sets contained in s (largest open set
contained in S).
Example. S = {(x, y) R2 |0 < z 1}. Int(s) are points with open -balls
belonging to S:
4.3
Continuity of functions
4.3.1
Continuous function at x0
76
4. Analysis
f (xn ) f (x0 ).
Example.
f (x) =
1 x>0
0 x0
4.3.2