AMath 584, Autumn 2017

Sample Solutions for Assignment 8.

Reading: Lectures 27-29 in the text.

1. p. 200, Exercise 26.1. Use the results of this exercise to show that if A is a normal
matrix and λ̃ is an eigenvalue of A + δA for some δA with kδAk2 ≤ , then there is an
eigenvalue λ of A such that |λ̃ − λ| ≤ . Would you say that the problem of computing
eigenvalues of a normal matrix is well-conditioned or ill-conditioned?
We must show that the following are equivalent:
(i) z is an eigenvalue of A + δA for some δA with kδAk2 ≤ .
(ii) There exists a vector u ∈ Cm with k(A − zI)uk2 ≤  and kuk2 = 1.
(iii) σm (zI − A) ≤ .
(iv) k(zI − A)−1 k2 ≥ −1 .
(i) ⇒ (ii). Let u be a normalized eigenvector of A + δA corresponding to
the eigenvalue z: (A + δA)u = zu ⇒ (A − zI)u = −(δA)u. Taking norms
on each side, k(A − zI)uk2 ≤ kδAk2 · kuk2 = .
(ii) ⇒ (iii). The smallest singular value σm (zI − A) satisfies

σm = min k(zI − A)vk2 .

kvk2 =1

Since kuk2 = 1 and k(A − zI)uk2 ≡ k(zI − A)uk2 ≤ , it follows that σm ≤ .

(iii) ⇒ (iv). Since the singular values of (zI − A)−1 are the inverses of the
singular values of zI − A, it follows that zI − A has a singular value that
is greater than or equal to −1 . Since k(zI − A)−1 k2 is the largest singular
value, it follows that k(zI − A)−1 k ≥ −1 .
(iv) ⇒ (i). Since k(zI − A)−1 k2 ≥ −1 , there is a vector v with kvk2 = 1
such that if w = (zI − A)−1 v, then kwk2 ≥ −1 . Multiplying each side by
(zI − A), we have (zI − A)w = v. It follows that
vw∗
 
zI − A − ∗ w = 0,
w w
so z is an eigenvalue of the matrix A + δA, where δA = (vw∗ )/(w∗ w). Since
kvk2 = 1 and kwk2 ≥ −1 , the 2-norm of δA is less than or equal to (kvk2 ·
kw∗ k2 )/(kw∗ k2 · kwk2 ) = 1/kwk2 ≤ .
If A is a normal matrix and λ̃ is an eigenvalue of A + δA for some δA with
kδAk2 ≤ , then by (iv), k(λ̃I −A)−1 k ≥ −1 . Writing the eigendecomposition
of A as A = QΛQ∗ , where Q is unitary, we can write

k(λ̃I − A)−1 k2 = kQ(λ̃I − Λ)−1 Q∗ k2 = k(λ̃I − Λ)−1 k2 ,

1
 and this last expression is just 1/ minj |λ̃ − λj |. Since this is greater than or
equal to −1 ,
min |λ̃ − λj | ≤ .
j

Since a change of size  in a normal matrix results in a change of size at most

 in the eigenvalues, I would say that the problem of finding the eigenvalues
of a normal matrix is well-conditioned.

2. Let    
0 1 0 0 1 0
A =  0 0 1 , A =  0 0 1  .
   

0 0 0  0 0
What are the eigenvalues of A and of A ? Would you say that the problem of computing
the eigenvalues of a matrix A of this form is well-conditioned or ill-conditioned?

The eigenvalues of A are all 0’s. The characteristic polynomial of A is

z 3 −, so its eigenvalues are the cube roots of : 1/3 , 1/3 ei2π/3 , and 1/3 ei4π/3 .
(Here I am assuming that  is real and positive. If  = ||eiθ , then replace
these values by ||1/3 eiθ/3 , ||1/3 ei(θ+2π)/3 , and ||1/3 ei(θ+4π)/3 .) The problem of
computing the eigenvalues of A is ill-conditioned because a change of order
 in A makes a much larger change in the eigenvalues: 1/3 ; for example, if
A is changed by  = 10−15 then the eigenvalues change by 10−5 . For an m
by m matrix of this form, the eigenvalues of A would be the mth roots of ;
so if  = 10−15 and m > 15, we expect no or at most one digit of accuracy.

3. p. 218, Exercise 28.1.

When you factor an orthogonal matrix Q in the form QR, R is just the
identity. When you then multiply the factors in reverse order, you just get
Q. Thus the unshifted QR algorithm makes no progress toward getting this
matrix to upper triangular (or, if symmetric, diagonal) form. This does not
contradict Theorem 28.4 because the eigenvalues all have magnitude 1; they
do not satisfy the hypothesis |λ1 | > . . . > |λm |.

4. Consider the matrix  

−1 2
 −1 . . . . . .
 

A= .
 
.. ..

 . . −1 

−1 2
Taking A to be a 10 by 10 matrix, try the following:

(a) What information does Gerschgorin’s theorem give you about this matrix?
The eigenvalues all lie in a disk of radius 2 about 2. Using the fact that
the matrix is symmetric, we also know that the eigenvalues are real. Thus
the eigenvalues all lie in [0, 4].

2
(b) Implement the power method to compute an approximation to the eigenvalue of
largest absolute value and its corresponding eigenvector. [Note: Use a random
initial vector. If you choose something special, like the vector of all 1’s, it may
turn out to be orthogonal to the eigenvector you are looking for.] Turn in a listing
of your code together with the eigenvalue/eigenvector pair that you computed.
Once you have a good approximate eigenvalue, look at the error in previous ap-
proximations and comment on the rate of convergence of the power method.
The following MATLAB code generates the output below. Looking at
ratios of errors in approximate eigenvalues, it looks like the error is being
multiplied by about .88 at each step. [This ratio is different early on, and
if one compares computed approximations to the final approximation,
then it appears to be different near the end; however, this is due to
the fact that the final approximation still is not as accurate as it could
be. Comparison of later approximations with the result generated by
MATLAB’s eig routine reveals error ratios of about .88.]

% Power method for computing eigenvalue of largest magnitude

% and corresponding eigenvector

% Set matrix.

n = 10;
A = 2*eye(n,n);
for i=1:n-1, A(i,i+1)=-1; A(i+1,i)=-1; end;

% Set random initial vector.

w = randn(n,1);
yk = w/norm(w);
eval_approx = [];

% Do first step to see if yk is close to an eigenvector

yktilde = A*yk;
lambdak = yk’*yktilde;
eval_approx = [eval_approx; lambdak];
resk = norm(yktilde - lambdak*yk);

% Iterate until residual is small

while resk > 1.e-6,

yk = yktilde/norm(yktilde);
yktilde = A*yk;
lambdak = yk’*yktilde;

3
 eval_approx = [eval_approx; lambdak];
resk = norm(yktilde - lambdak*yk);
end;

lambdak, yk % Print out eigenpair

lambdak =

3.9190

yk =

-0.1201
0.2305
-0.3223
0.3879
-0.4221
0.4221
-0.3879
0.3223
-0.2305
0.1201

(c) Implement the QR algorithm (without shifts) to take A to diagonal form. You may
use Matlab routine [Q,R] = qr(A); to perform the necessary QR decompositions.
Comment on the rate at which the off-diagonal entries in A are reduced.
The following MATLAB code produces the results below. The algorithm
required 215 steps to reduce the Frobenius norm of the off-diagonal part
below 10−6 . It also shows (in subdiag_ratios) the factor by which
each subdiagonal entry was reduced over several steps. The (2, 1) entry
was the slowest to decay, being multiplied by about 0.9397 at each step;
the (n, n − 1) entry was the fastest to decay, being multiplied by about
0.2552 at each step. These ratios match the eigenvalue ratios; that is,
the (j + 1, j) entry was multiplied by about the factor |λj+1 /λj | at each
step.

% QR algorithm

% Set matrix.

n = 10;
A = 2*eye(n,n);
for i=1:n-1, A(i,i+1)=-1; A(i+1,i)=-1; end;

4
% Iterate until matrix is close to diagonal.

Ak = A; k = 0;
subdiag(:,1) = diag(Ak,-1); % Keep track of size of subdiag
while norm(Ak-diag(diag(Ak)),’fro’) > 1.e-6,
[Qk,Rk] = qr(Ak);
Ak = Rk*Qk;
k = k+1;
subdiag(:,k+1) = diag(Ak,-1);
end;

evals = diag(Ak), k % Print out approx evals

% and no. of iterations
% Print out ratios of subdiagonal entries over a few steps.
% Also print out eigenvalue ratios.
subdiag_ratios = abs(subdiag(:,k-5:k) ./ subdiag(:,k-6:k-1))
evals_sorted = sort(abs(diag(Ak)),’descend’);
eval_ratios = evals_sorted(2:n) ./ evals_sorted(1:n-1)

evals =

3.9190
3.6825
3.3097
2.8308
2.2846
1.7154
1.1692
0.6903
0.3175
0.0810

k =

215

subdiag_ratios =

0.9397 0.9397 0.9397 0.9397 0.9397 0.9397

0.8988 0.8988 0.8988 0.8988 0.8988 0.8988

5
 0.8553 0.8553 0.8553 0.8553 0.8553 0.8553
0.8071 0.8071 0.8071 0.8071 0.8071 0.8071
0.7508 0.7508 0.7508 0.7508 0.7508 0.7508
0.6816 0.6816 0.6816 0.6816 0.6816 0.6816
0.5904 0.5904 0.5904 0.5904 0.5904 0.5904
0.4599 0.4599 0.4599 0.4599 0.4599 0.4599
0.2552 0.2552 0.2552 0.2552 0.2552 0.2552

eval_ratios =

0.9397
0.8988
0.8553
0.8071
0.7508
0.6816
0.5904
0.4599
0.2552

(d) Take one of the eigenvalues computed in your QR algorithm and, using it as a
shift in inverse iteration, compute the corresponding eigenvector.
The following code computes the eigenvector corresponding to the fifth
eigenvalue returned by the QR algorithm in (c). It requires only 1 itera-
tion and produces the results below.

% First run QR algorithm in ex12_5c.m

% Then run this code using inverse iteration
% to compute the eigenvector corresponding to
% the fifth eigenvalue.

s = Ak(5,5); % Fifth diagonal entry of final matrix

% from QR.

w = randn(n,1); % Random initial vector.

yk = w/norm(w);
k = 0;
resk = norm(A*yk - yk*s);
while resk > 1.e-6 % Iterate until resid <= 1.e-6
yktilde = (A - s*eye(n,n))\yk;
yk = yktilde/norm(yktilde);
k = k+1;
resk = norm(A*yk - yk*s);

6
end;

% Print results.
s, yk, k

s =

2.2846

yk =

-0.4221
0.1201
0.3879
-0.2305
-0.3223
0.3223
0.2305
-0.3879
-0.1201
0.4221

k =