SCS1113: COMPUTATIONAL

MATHEMATICS

CHAPTER 7
EIGEN VALUE
&
EIGEN VECTOR
nzah@utm.my: 2013/2014 - 2
1

Introduction
Let A be a square matrix of
dimension nxn and let v be a vector
of dimension n.
The product,
Y=Av
can be viewed as a linear
transformation from n-dimensional
into itself.
2

Introduction

(cont.)

We want to find scalars ( ) for

which there exists a nonzero vector
v such that
Av = v
that is, the linear transformation
T(v) = Av maps v onto the multiple
v.
When this occurs, we call v an eigen
vector that corresponds to the eigen
value (), and together they form
3

Introduction

(cont.)

The identity matrix [I] can be used to

express equation
Av = v
as
Av = Iv
which is then rewritten in the
standard form for a linear system as,
(A - I) v = 0
4

Introduction

(cont.)

This linear system,

(A - I) v = 0
has nontrivial solutions if and only if
the matrix A - I is singular, that is
|A - I| = 0
When the determinant is expanded,
it becomes a polynomial of degree n,
which is called the characteristic
polynomial,
p()=|A - I|
5

Eigen Value & Eigen Vector

If A is an nxn matrix, then its n eigen
values 1, 2,, n are the roots of the
characteristic polynomial
p()= det (A - I)
If is an eigen value of A and the
nonzero vector v has the property
that
Av = v
then v is called an eigen vector of
A corresponding to the eigen value .
6

Example 7.1
Let

3 6
A

1
4

3 6
1 0 3
A I

1 4
0 1 1

The characteristic polynomial of the

matrix A
2

P1 ( ) det A I 7 6 ( 6)( 1)

Therefore, the eigen values of A are =

6 and
7

Example 7.2
Let,

Then,

1 6 12
B 0 13 30
0 9 20

1
B I 0
0

6
13
9

12
30
20

The characteristic polynomial of the

3
2
matrix
B
P ( ) det B I 6 3 10 0

( 1)( -5)( -2)= 0

The eigen values are = -1; = 5 and
8

Eigen Values
To find eigen values when the
dimension n is small:
Find the coefficient of the characteristic
polynomial
Finds its root
Find the nonzero solutions of the
homogeneous linear system (A - I) v = 0

Finding the determinant of an nxn

matrix is computationally expensive
and finding good approximations to
the roots of p() is also difficult.
9

Theorem 7.1
Assume that A is nxn matrix and let

i , i 1,2,..., n
are eigen values of A
(1)

a
i 1

i 1

ii

2)

i 1

10

Gerschgorins Circle
Theorem
Assume that A is nxn matrix, and let
Bi denote the disk in the complex
plane with center aii and radius ri
n

ri aij
j 1

i j

Bi aii ri
n

S i 1 Bi

11

Example 7.3
Let

3 2 1

A 1 3
1
1 2 4

12

Example 7.3
Row 1

(cont.)

center
radius

3 2 1

A 1 3
1
1 2 4

3 2 1
3 3 3 3 0,6
0 6

(0,6)
13

Example 7.3

(cont.)

Circle
radius, 3

center

14

Example 7.3
Row 2

(cont.)
center
radius

3 2 1

A 1 3
1
1 2 4

3 1 1
3 2
1 5

(1,5)
15

Example 7.3

(cont.)

Circle
radius, 2

0 1

center

16

Example 7.3
Row 3

(cont.)

center
radius

3 2 1

A 1 3
1
1 2 4

( 4 ) 1 2
4 3
7 1
(7,1)
17

Example 7.3

(cont.)

Circle
radius, 3
0
-7

-4

-1

center

18

Example 7.3

(cont.)

( 7,6)

-7

-1

19

Power Method
The power method is an iterative
technique used to determine the
dominant eigen value of a matrix
that is, the eigen value with the
largest magnitude.
By modifying the method slightly, it
can also be used to determine other
eigen values.
One useful feature of the power
method is that it produces not only
an eigen value, but also the
20

Power Method

(cont.)

If 1 is an eigen value of A that is

larger in absolute value than any
other eigen value, it is called the
dominant eigen value.
An eigen vector v1 corresponding to
1 is called a dominant eigen
vector.
21

Power Method

(cont.)

To apply the power method, we

assume that the nxn matrix A has n
eigen values 1, 2,, n with an
associated collection of linearly
independent eigenvectors {v(1),
v(2),.. v(n)}

1 2 3 ... n

We assume that A has precisely one

eigen value, 1 that is largest in
magnitude, so that
22

Power Method Algorithm

Av v

1
v Av

Start with vector, v(0)

v

(1)

Av ( 0 )
m1

23

Power Method Algorithm

(cont.)

Generate the sequence {v(k)}

recursively, using
v

( k 1)

Av ( k )
mk 1

k 0,1,2....

Where mk+1 is the coordinate of Av(k)

of largest magnitude (in the case of a
tie, choose the coordinate that
comes first).

24

Power Method Algorithm

(cont.)

The sequences {v(k)} and {mk} will

converge to v and , respectively.
lim vk v
k

lim mk
k

Termination criteria: Select a

tolerance (error), >0 and
generate( k v1) 1,.,v
(k )
k+1 until
v

25

Example 7.4
Let

1 2 1
A 1 0
1
4 4 5

Use the Power method to

approximate the most dominant
eigen value of the matrix. Let
v(0)=(0,0,1)T and iterate until
=0.001.
26

Example 7.4 - Solution

v Av

(1)

( 0)

1 2 1 0 1

1 0
1 0 1
4 4 5 1 5

* Av ( 0 )
m1

1 0.2
v 1 / 5 1 0.2
m1
5 1.0

m1 5

(largest
magnitud
e)
v (1) v ( 0 )
(next
iteration)

27

Example 7.4 - Solution

v Av (1)

( 2)

1 2 1
1 0
1
4 4 5

0.2
0.8
0.2 0.8

1.0
3.4

(cont.)

m2 3.4

0.8
0.235
v / m2 1 / 3.4 0.8 0.235
3.4
1.0

v ( 2 ) v (1)
(next
iteration)

28

Example 7.4 - Solution

k
0

(v ( k ) ) T
0
1

(cont.)

( Av ( k ) ) T
1

mk 1
5
3 .4
3.12
3.04

1 0 .2
2 0.235
3 0.245

0 .2
0.235
0.245

1 0 .8
1 0.765
1 0.755

0 .8
0.765
0.755

3 .4
3.12
3.04

4 0.248
5 0.249

0.248
0.249

1 0.752
1 0.751

0.752
0.751

3.016 3.016
3.008 3.008

6 0.250

0.250

1 0.750

0.750

3.000 3.000

7 0.250

0.250

1
29

Example 7.4 - Solution

v ( 7 ) v ( 6)

1 m7 3.000
v1 v ( 7 )

0.250
0.250
1.000

30

(cont.)

Example 7.4 - Solution

Av 1 1 v 1

1 0.25
1 2
1 0
0.25 3.000
1

4 4 5 1.0
0.75 0.75
0.75 0.75

3.00 3.00

0.25
0.25

1.0

31

(cont.)

Exercise 7.1
Let

3 2 1

A 1 2 3
2 0

(0)

[0.1,0.4,1]

Use the Gerschgorins Circle

Theorem to determine a region
containing all the eigen values of A.
Use the Power method to
determine the dominant eigen value
of the matrix and its associated
eigenvector. Iterate until =0.005.
32

To be continued

The Shifted Power Method

Consider the standard eigenvalue
problem
Av=v
By subtracting a scalar s from both
sides of the standard eigenvalue
problem, the eigenvalues of the
matrix are shifted:
Av sIv =v sv
which yields (A sI) v = ( s)v
34

The Shifted Power Method

(cont.)

Ashifted is the shifted matrix,

Ashifted = (A sI)
shifted = s is the eigenvalue of the
shifted matrix.
Shifting matrix A by a scalar s shifts
the eigenvalue by s.
Shifting a matrix by a scalar does not
affect the eigenvectors.
35

The Shifted Power Method

(cont.)

Shifting of eigen values of a matrix

can be used to find the opposite
extreme eigen value, which is either
the smallest magnitude eigen
value, or
the largest magnitude eigen
value of opposite sign.

36

Opposite Extreme Eigen

value
Consider a matrix whose eigen
values are all the same sign; =1, 2,
4, and 8.
= 8 is the eigen value of largest
magnitude.
=1 is the opposite extreme eigen
value.
The eigen value of largest magnitude
1=8 can be found by the power
37

Opposite Extreme Eigen

value
The largest magnitude eigen value of
the shifted matrix can be found by the
power method; shifted, largest = -7.
The smallest eigen value of the
original matrix may be found by
smallest = shifted, largest + 8 = -7+ 8
=1
38

The Shifted Power Method

Consider a matrix whose eigen
values are both positive and
negative; =-1, -2, 4, and 8.
=8 is the eigen value of largest
magnitude.
=-2 is the opposite extreme eigen
value.
The eigen value of largest magnitude
1=8 can be found by the power
method.
39

The Shifted Power Method

(cont.)

The largest magnitude eigen value of

the shifted matrix can be found by
the power method; shifted, largest = -10.
The largest magnitude eigen value of
opposite sign of the original matrix
may be found by
largest,negative = shifted, largest + 8 =
-10+ 8 = -2
40

The Shifted Power Method

(cont.)

The shifted power method can be

summarized as follows:
1. Solve for the eigen value of largest
magnitude 1 using the power method.
2. Shift the matrix A by s =1 to obtain the
shifted matrix Ashifted.
3. Solve for the eigen value shifted by the
power method.
4. Compute the opposite extreme eigen value
of matrix A by = shifted + s
41

Example 7.5
Let,

1 2 1
A 1 0
1
4 4 5

The dominant eigen value of A is 1=

3.0.
Use the shifted power method to find
the remaining eigen values of A.

42

Example 7.5 - Solution

Let B = Ashifted
B A 1 I A 3I
1 2 1
1 0 0
2 2 1
1 0
3 0 1 0 1 3 1
1

4 4 5
0 0 1
4 4 2

43

Example 7.5 - Solution

Let, v

v Bv

(0)

(0)

(cont.)

=[0, 1, 0]T and =0.001

2 2 1 0 2

1 3 1 1 3
4 4 2 0 4

m1 4

44

Example 7.5 - Solution

v (1)

0 .5
v / m1 0.75
1.0

v (1) v ( 0)
(next iteration)

45

(cont.)

(v ( k ) ) T

Example
1
0

( Bv ( k ) ) T

m k 1

1.5

1.75

0.5

0.75

0.5

0.583

1 1.166

1.249

2.332 2.332

0.5

0.536

1 1.072

1.108

2.144 2.144

0.5

0.517

1 1.034

1.051

2.068 2.068

0.5

0.508

1 1.016

1.024

2.032 2.032

6
7

0.5
0.5

0.504
0.502

1 1.008
1 1.004

1.012
1.006

2.016 2.016
2.008 2.008

0.5

0.501

1 1.002

1.003

2.004 2.004

0.5

0.5

1 1.000

1.000

2.000 2.000

10 0.5

0.5

46

Example 7.5 - Solution

(cont.)

The largest magnitude eigen value of

the shifted matrix and its
eigenvector,
T
v [0.5,0.5,1.0]

m 2.0
shifted

10

The opposite extreme eigen value of

matrix
A
and
its
eigenvector,
2.0 3.0 1.0
shifted

v [0.5,0.5,1.0]T
47

Example 7.5 - Solution

Av v
1 0.5
1 2
1 0
0 .5 1 .0
1

4 4 5 1.0

0 .5
0 .5

1.0

0.5 0.5
0.5 0.5

1.0 1.0

48

(cont.)

Example 7.5 - Solution

(cont.)

Since the opposite extreme

eigenvalue =1, has the same sign
as the largest magnitude eigen value
of the original matrix 1=3, all eigen
values of A are positive and =1 is
the smallest eigen value of matrix A.

49

Example 7.5 - Solution

(cont.)

Matrix A has 3 eigen values, 1,2,

and 3.
Let,
|1| |2| |3|
The largest eigen value of matrix A,
1=3
The smallest eigen value of matrix A,
3=1
50

Example 7.5 - Solution

(cont.)

Find the intermediate eigen value of

matrix A, 2
3

a
i 1

i 1

ii

1 2 3 a11 a 22 a33 1 0 5 6
3 2 1 a11 a22 a33 6

2 6 1.0 3.0 2.0

51

Exercise 7.2
Let,

19 20 6

D 12 13 3
30 30 12

The dominant eigenvalue of D is

1=3.0.
Use the shifted power method to
find the remaining eigenvalues of D.
Let v (0) =[1, 0.5, 0]T and iterate until
=0.01.
52