Académique Documents
Professionnel Documents
Culture Documents
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Outline
Scientific Computing: An Introductory Survey
Chapter 4 Eigenvalue Problems
Eigenvalue Problems
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
1 / 87
Eigenvalue Problems
Eigenvalues and Eigenvectors
Geometric Interpretation
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Eigenvalue Problems
Scientific Computing
2 / 87
Eigenvalue Problems
Eigenvalues and Eigenvectors
Geometric Interpretation
Ax = x
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
3 / 87
Eigenvalue Problems
Eigenvalues and Eigenvectors
Geometric Interpretation
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Geometric Interpretation
Scientific Computing
4 / 87
Eigenvalue Problems
Eigenvalues and Eigenvectors
Geometric Interpretation
where i = 1
A=
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
5 / 87
Characteristic Polynomial
Relevant Properties of Matrices
Conditioning
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Characteristic Polynomial
Scientific Computing
6 / 87
Characteristic Polynomial
Relevant Properties of Matrices
Conditioning
Equation Ax = x is equivalent to
Characteristic polynomial of previous example matrix is
3 1
1 0
det
=
1
3
0 1
(A I)x = 0
which has nonzero solution x if, and only if, its matrix is
singular
Eigenvalues of A are roots i of characteristic polynomial
det
det(A I) = 0
in of degree n
Fundamental Theorem of Algebra implies that n n matrix
A always has n eigenvalues, but they may not be real nor
distinct
Complex eigenvalues of real matrix occur in complex
conjugate pairs: if + i
is eigenvalue of real matrix, then
so is i, where i = 1
Michael T. Heath
Scientific Computing
3 1
=
1 3
(3 )(3 ) (1)(1) = 2 6 + 8 = 0
so eigenvalues are given by
6 36 32
=
, or 1 = 2,
2
7 / 87
Michael T. Heath
Scientific Computing
2 = 4
8 / 87
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Characteristic Polynomial
Relevant Properties of Matrices
Conditioning
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Companion Matrix
Characteristic Polynomial
Relevant Properties of Matrices
Conditioning
Monic polynomial
p() = c0 + c1 + + cn1 n1 + n
Computing eigenvalues using characteristic polynomial is
not recommended because of
0 0 0 c0
1 0 0 c1
Cn = 0 1 0 c2
.. .. . .
..
..
. .
. .
.
0 0 1 cn1
Scientific Computing
9 / 87
Characteristic Polynomial
Relevant Properties of Matrices
Conditioning
A=
mach
det(A I) = 2 2 + (1 2 ) = 2 2 + 1
which has 1 as double root
X 1 AX = D
10 / 87
Characteristic Polynomial
Relevant Properties of Matrices
Conditioning
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
1
1
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
Characteristic Polynomial
Relevant Properties of Matrices
Conditioning
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
12 / 87
Characteristic Polynomial
Relevant Properties of Matrices
Conditioning
Property
diagonal
tridiagonal
triangular
Hessenberg
Eigenspace = S = {x : Ax = x}
Subspace S of Rn (or Cn ) is invariant if AS S
orthogonal
unitary
symmetric
Hermitian
normal
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
13 / 87
Characteristic Polynomial
Relevant Properties of Matrices
Conditioning
Transpose:
1 2
3 4
T
=
Conjugate transpose:
1 3
2 4
Scientific Computing
14 / 87
Characteristic Polynomial
Relevant Properties of Matrices
Conditioning
0 1
1 0
2/2 2/2
,
,
1 0
0 1
2/2
2/2
i2/2
2/2
Unitary:
2/2 i 2/2
1 1
Nonorthogonal:
1 2
1 2 0
Normal: 0 1 2
2 0 1
1 1
Nonnormal:
0 1
Orthogonal:
1 2
2 3
1 3
Nonsymmetric:
2 4
1
1+i
Hermitian:
1i
2
1
1+i
NonHermitian:
1+i
2
Symmetric:
Scientific Computing
Michael T. Heath
Examples, continued
H
1 + i 1 + 2i
1i 2+i
=
1 2i 2 + 2i
2 i 2 2i
Michael T. Heath
AT A = AAT = I
AH A = AAH = I
A = AT
A = AH
AH A = AAH
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Definition
aij = 0 for i 6= j
aij = 0 for |i j| > 1
aij = 0 for i > j (upper)
aij = 0 for i < j (lower)
aij = 0 for i > j + 1 (upper)
aij = 0 for i < j 1 (lower)
15 / 87
Michael T. Heath
Scientific Computing
16 / 87
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Characteristic Polynomial
Relevant Properties of Matrices
Conditioning
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Characteristic Polynomial
Relevant Properties of Matrices
Conditioning
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
17 / 87
Characteristic Polynomial
Relevant Properties of Matrices
Conditioning
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Conditioning of Eigenvalues
|| /
Problem Transformations
Power Iteration and Variants
Other Methods
20 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Then
By = y T 1 AT y = y A(T y) = (T y)
21 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Scientific Computing
22 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Diagonal Form
Eigenvalues of diagonal matrix are diagonal entries, and
eigenvectors are columns of identity matrix
and hence
0.5
0.5
3 1 1
1
2 0
=
0.5 0.5 1
3 1 1
0 4
Scientific Computing
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Michael T. Heath
Scientific Computing
Similarity Transformation
Michael T. Heath
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Problem Transformations
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
kyk2 kxk2
1
kEk2 =
kEk2
|y H x|
cos()
Michael T. Heath
18 / 87
Characteristic Polynomial
Relevant Properties of Matrices
Conditioning
Conditioning of Eigenvalues
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
23 / 87
Michael T. Heath
Scientific Computing
24 / 87
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Problem Transformations
Power Iteration and Variants
Other Methods
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Triangular Form
Problem Transformations
Power Iteration and Variants
Other Methods
U11 u U13
0 vT
A I = 0
O 0 U33
If
App
(A) =
Scientific Computing
25 / 87
arbitrary
unitary
arbitrary
nonsingular
Michael T. Heath
diagonal
real diagonal
real diagonal
diagonal
real block triangular
(real Schur)
upper triangular
(Schur)
almost diagonal
(Jordan)
Simplest method for computing one eigenvalueeigenvector pair is power iteration, which repeatedly
multiplies matrix times initial starting vector
Assume A has unique eigenvalue of maximum modulus,
say 1 , with corresponding eigenvector v1
Then, starting from nonzero vector x0 , iteration scheme
xk = Axk1
converges to multiple of eigenvector v1 corresponding to
dominant eigenvalue 1
27 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
28 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
i=1
Scientific Computing
Michael T. Heath
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
26 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Power Iteration
Scientific Computing
i=1
Scientific Computing
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
(Ajj )
Problem Transformations
Power Iteration and Variants
Other Methods
A
distinct eigenvalues
real symmetric
complex Hermitian
normal
arbitrary real
p
[
j=1
is corresponding eigenvector
Michael T. Heath
A1p
A2p
..
.
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
A11 A12
A22
A=
..
29 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
30 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
yk = Axk1
xk = yk /kyk k
Michael T. Heath
Scientific Computing
31 / 87
Michael T. Heath
Scientific Computing
32 / 87
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Problem Transformations
Power Iteration and Variants
Other Methods
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Geometric Interpretation
Behavior of power iteration depicted geometrically
xTk
0.000
0.333
0.600
0.778
0.882
0.939
0.969
0.984
0.992
kyk k
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.500
1.667
1.800
1.889
1.941
1.970
1.985
1.992
Problem Transformations
Power Iteration and Variants
Other Methods
Scientific Computing
33 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
34 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
so convergence is accelerated
Shift must then be added to result to obtain eigenvalue of
original matrix
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
35 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Inverse Iteration
Scientific Computing
36 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Ayk = xk1
xk = yk /kyk k
which is equivalent to power iteration applied to A1
Inverse of A not computed explicitly, but factorization of A
used to solve system of linear equations at each iteration
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
37 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
38 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Scientific Computing
39 / 87
Michael T. Heath
Scientific Computing
40 / 87
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Problem Transformations
Power Iteration and Variants
Other Methods
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Rayleigh Quotient
Problem Transformations
Power Iteration and Variants
Other Methods
k
0
1
2
3
4
5
6
Scientific Computing
41 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
xTk
0.000
0.333
0.600
0.778
0.882
0.939
0.969
1.0
1.0
1.0
1.0
1.0
1.0
1.0
kyk k
1.500
1.667
1.800
1.889
1.941
1.970
1.500
1.800
1.941
1.985
1.996
1.999
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
42 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
(A k I)yk+1 = xk
xk+1 = yk+1 /kyk+1 k
starting from given nonzero vector x0
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
43 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
xTk
0.807 0.397
0.924 1.000
1.000 1.000
Scientific Computing
44 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Deflation
After eigenvalue 1 and corresponding eigenvector x1
have been computed, then additional eigenvalues
2 , . . . , n of A can be computed by deflation, which
effectively removes known eigenvalue
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
k
1.896
1.998
2.000
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
45 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Deflation, continued
Scientific Computing
46 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Deflation, continued
Alternative approach lets u1 be any vector such that
uT1 x1 = 1
x2 = H 1
, where =
y2
2 1
Michael T. Heath
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
47 / 87
Michael T. Heath
Scientific Computing
48 / 87
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Problem Transformations
Power Iteration and Variants
Other Methods
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Simultaneous Iteration
Problem Transformations
Power Iteration and Variants
Other Methods
Orthogonal Iteration
As with power iteration, normalization is needed with
simultaneous iteration
Simplest method for computing many eigenvalueeigenvector pairs is simultaneous iteration, which
repeatedly multiplies matrix times matrix of initial starting
vectors
k
Xk = AQ
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
49 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
QR Iteration
Scientific Computing
50 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
QR Iteration, continued
H AQ
k
Ak = Q
k
Ak = Rk Qk = QH
k Ak1 Qk
Scientific Computing
51 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Example: QR Iteration
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
52 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
7 2
2 4
Compute QR factorization
.962 .275 7.28 3.02
A 0 = Q1 R 1 =
.275
.962
0
3.30
Let A0 =
A k = R k Qk + k I
7.83 .906
A1 = R1 Q1 =
.906 3.17
Scientific Computing
53 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
54 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Scientific Computing
Preliminary Reduction
Michael T. Heath
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
55 / 87
Michael T. Heath
Scientific Computing
56 / 87
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Problem Transformations
Power Iteration and Variants
Other Methods
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Problem Transformations
Power Iteration and Variants
Other Methods
tridiagonal
diagonal
triangular
or
general
Hessenberg
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
57 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Cost of QR Iteration
Scientific Computing
58 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
then
Kn1 AKn = Cn
General matrices
10n3 for eigenvalues only
25n3 for eigenvalues and eigenvectors
so that
1
QH
n AQn = Rn Cn Rn H
Scientific Computing
59 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
60 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
j = 1, . . . , k
Scientific Computing
Arnoldi Iteration
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
61 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
62 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
If
Qk = q 1
qk
then
Hk = QH
k AQk
is upper Hessenberg matrix
Scientific Computing
63 / 87
Michael T. Heath
Scientific Computing
64 / 87
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Problem Transformations
Power Iteration and Variants
Other Methods
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Lanczos Iteration
Problem Transformations
Power Iteration and Variants
Other Methods
q0 = 0
0 = 0
x0 = arbitrary nonzero starting vector
q1 = x0 /kx0 k2
for k = 1, 2, . . .
uk = Aqk
k = qkH uk
uk = uk k1 qk1 k qk
k = kuk k2
if k = 0 then stop
qk+1 = uk /k
end
Michael T. Heath
Scientific Computing
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
65 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
Michael T. Heath
Scientific Computing
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
66 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
67 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
68 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Jacobi Method
One of oldest methods for computing eigenvalues is Jacobi
method, which uses similarity transformation based on
plane rotations
Sequence of plane rotations chosen to annihilate
symmetric pairs of matrix entries, eventually converging to
diagonal form
Choice of plane rotation slightly more complicated than in
Givens method for QR factorization
To annihilate given off-diagonal pair, choose c and s so that
c s a b
c s
J T AJ =
s
c b d s c
2
c a 2csb + s2 d
c2 b + cs(a d) s2 b
=
2
2
2
2
c b + cs(a d) s b
c d + 2csb + s a
is diagonal
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
69 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Consider 2 2 matrix
c2 b + cs(a d) s2 b = 0
Dividing both sides by
A=
we obtain
s (a d) s2
2 =0
1+
c
b
c
Making substitution t = s/c, we get quadratic equation
(a d)
t2 = 0
b
for tangent t of
angle of rotation, from which we can
recover c = 1/( 1 + t2 ) and s = c t
1/2 1/2 1 2 1/2 1/2
3
0
J T AJ =
=
0 1
1/ 2 1/ 2 2 1 1/ 2
1/ 2
1 2
2 1
1+t
Michael T. Heath
70 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
c2 b,
Scientific Computing
71 / 87
Michael T. Heath
Scientific Computing
72 / 87
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Problem Transformations
Power Iteration and Variants
Other Methods
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
73 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
74 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
0.888 0.460 0
0.888 0
J1 = 0.460
0
0
1
0.707 0 0.707
0
1
0
J0 =
0.707 0
0.707
to obtain
to obtain
A2 =
J1T A1 J1
3
0.707
0
T
2
0.707
A1 = J0 A0 J0 = 0.707
0
0.707 1
Michael T. Heath
Scientific Computing
Example, continued
1 0 2
Let A0 = 0 2 1
2 1 1
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Problem Transformations
Power Iteration and Variants
Other Methods
Scientific Computing
75 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
3.366
0
0.325
1.634 0.628
= 0
0.325 0.628 1
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Example, continued
Scientific Computing
76 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Example, continued
Beginning new sweep, again annihilate (1,3) and (3,1)
entries using rotation
0.998 0 0.070
1
0
J3 = 0
0.070 0
0.998
1
0
0
J2 = 0 0.975 0.221
0 0.221
0.975
to obtain
to obtain
3.366 0.072
0.317
0
A3 = J2T A2 J2 = 0.072 1.776
0.317
0
1.142
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
A4 =
77 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
J3T A3 J3
3.388
0.072
0
1.776 0.005
= 0.072
0
0.005 1.164
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Example, continued
Scientific Computing
78 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Bisection or Spectrum-Slicing
For real symmetric matrix, can determine how many
eigenvalues are less than given real number
Scientific Computing
79 / 87
Michael T. Heath
Scientific Computing
80 / 87
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Problem Transformations
Power Iteration and Variants
Other Methods
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Sturm Sequence
Problem Transformations
Power Iteration and Variants
Other Methods
Divide-and-Conquer Method
Scientific Computing
81 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
Ax = Bx
where A and B are given n n matrices
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
82 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
83 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
QZ Algorithm
Scientific Computing
84 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
QZ Algorithm, continued
Michael T. Heath
Eigenvalue Problems
Existence, Uniqueness, and Conditioning
Computing Eigenvalues and Eigenvectors
Scientific Computing
85 / 87
Problem Transformations
Power Iteration and Variants
Other Methods
Computing SVD
Singular values of A are nonnegative square roots of
eigenvalues of AT A, and columns of U and V are
orthonormal eigenvectors of AAT and AT A, respectively
Algorithms for computing SVD work directly with A, without
forming AAT or AT A, to avoid loss of information
associated with forming these matrix products explicitly
SVD is usually computed by variant of QR iteration, with A
first reduced to bidiagonal form by orthogonal
transformations, then remaining off-diagonal entries are
annihilated iteratively
SVD can also be computed by variant of Jacobi method,
which can be useful on parallel computers or if matrix has
special structure
Michael T. Heath
Scientific Computing
87 / 87
Michael T. Heath
Scientific Computing
86 / 87