Krylov 2 Filimon

Krylov Subspace Iteration Methods
Anastasia Filimon
ETH Zurich
29 May 2008
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 1 / 24
Set up
The methods represent iterative techniques for solving large linear
systems
Ax = b,
where A is non-singular nxn-matrix, b is n-vector, n is large.
They are based on projection processes onto Krylov subspaces.
Krylov Subspaces
Krylov subspace generated by an nxn-matrix A, and an n-vector b is
the subspace spanned by the vectors of the Krylov sequence:
K
m
= spanfb, Ab, A
2
b, ..., A
m1
bg.
The projection method seeks an approximate solution x
m
from an
ane subspace x
0
+K
m
by imposing condition b Ax
m
? L
m
, L
m
is
another subspace of dimension m, x
0
is an initial guess to the
solution. In the case of Krylov subspace methods K
m
= K
m
(A, r
0
),
r
0
= b Ax
0
is an n-vector
K
m
= spanfr
0
, Ar
0
, A
2
r
0
, ..., A
m1
r
0
g.
Property 1. K
m
is the subspace of all vectors in R
n
which can be
written as
x = p(A)v,
where p is a polynomial of degree not exceeding m 1.
Krylov Subspaces
The dierent versions of Krylov subspace methods arise from dierent
choices of the subspace L
m
and from the ways in which the system is
preconditioned. Two broad choices for L
m
give rise to the best-known
techniques:
L
m
= K
m
FOM
L
m
= AK
m
GMRES, MINRES.
Arnoldi Modied Gram-Schmidt Method
This is an algorithm for building an orthogonal basis of the Krylov
subspace K
m
.
At each step, the algorithm multiplies Arnoldi vector v
j
by A and then
orthonormalizes the resulting vector w
j
against all previous v
j
s by a
standard Gram-Schmidt procedure.
Arnoldis Method
Proposition 1. Assume the Arnoldi Algorithm does not stop before
the m-th step. Then the vectors v
1
, v
2
, ..., v
m
form an orthonormal
basis of the Krylov subspace
K
m
= spanfv
1
, Av
1
, ..., A
m1
v
1
g.
Proposition 2. Projection method onto the subspace K
j
will be
exact when a breakdown occurs at step j .
Proposition 3. Denote by V
m
the nxm matrix with column vectors
v
1
, ..., v
m
, by H
m
, the (m + 1)xm Hessenberg matrix whose nonzero
entries h
ij
are dened by Arnoldi Modied Gram-Schmidt Algorithm
and by H
m
the matrix obtained from H
m
by deleting its last row.
Then the following relations hold:
AV
m
= V
m
H
m
+w
m
e
T
m
= V
m+1
H
m
V
T
m
AV
m
= H
m
.
Full Orthogonalization Method (FOM)
Given the initial guess x
0
to the original linear system Ax = b,
consider the projection method described before, which takes
L
m
= K
m
(A, r
0
), where r
0
= b Ax
0
. If v
1
=
r
0
kr
0
k
2
in Arnoldis
method and set = kr
0
k
2
, then
V
T
m
AV
m
= H
m
from Proposition 3, and
V
T
m
r
0
= V
T
m
(v
1
) = e
1
.
As a result, the approximate solution using the above mdimensional
subspaces is given by
x
m
= x
0
+V
m
y
m
, y
m
= H
1
m
(e
1
)
The presented algorithm depends on a parameter m which is the
dimension of the Krylov subspace. In practice it is desirable to select
m in a dynamic fashion. This would be possible if the residual norm
of x
m
is available without compution x
m
itself.
Proposition 4. The residual vector of the approximate solution x
m
computed by the FOM Algorithm is such that
kb Ax
m
k
2
= h
m+1,m
e
T
m
y
m
.
Generalized Minimum Residual Method (GMRES)
The method is a projection method based on taking L
m
= AK
m
,in
which K
m
is the m-th Krylov subspace with v
1
= r
0
/ kr
0
k
2.
Such a
technique minimizes the residual norm over all vectors in x
0
+K
m
.
The implementation of an algorithm based on this approach is similar
to that of the FOM algorithm.
Any vector x in x
0
+K
m
can be written as x = x
0
+V
m
y, where y is
an m-vector.
Dene
J(y) = kb Axk
2
= kb A(x
0
+V
m
y)k
2
,
Using relation from Proposition 3
b Ax = b A(x
0
+V
m
y) = r
o
AV
m
y = v
1
V
m+1
H
m
y
= V
m+1
(e
1
H
m
y).
Since the column-vectors of V
m+1
are orthonormal, then
J(y) = kb A(x
0
+V
m
y)k
2
=
e
1
H
m
y
2
.
The GMRES approximation is the unique vector of x
0
+K
m
which
minimizes J(y), i.e.
x
m
= x
0
+V
m
y
m
, where
y
m
= arg min
y
e
1
H
m
y
2
The minimizer is inexpensive to compute since it requires the solution
of an (m + 1)xm least-squares problem where m is typically small.
Generalized Minimum Residual Method (GMRES).
Practical Implementation issues
A clear diculty with GMRES algorithm is that it does not provide
the approximate solution x
m
explicitly at each step. As a result, it is
not easy to determine when to stop. However, there is a solution
related to the way in which the least-squares problem is solved.
In order to solve the least-squares problem min
e
1
H
m
y
, it is
natural to transform the Hessenberg matrix into upper triangular form
by using plane rotations.
Proposition 5. Dene the rotation matrices to transform H
m
into
upper triangular form
i
=
0
B
B
B
B
B
B
B
B
B
B
@
1
...
1
c
i
s
i
s
i
c
i
1
...
1
1
C
C
C
C
C
C
C
C
C
C
A
c
2
i
+s
2
i
= 1, s
i
=
h
i +1,i
q
(h
(i 1)
ii
)
2
+h
2
i +1,i
, c
i
=
h
(i 1)
ii
q
(h
(i 1)
ii
)
2
+h
2
i +1,i
.
Dene the product of matrices
i
Q
m
=
m
m1
...
1
,
R
m
, g
m
= (
1
, ...,
m+1
)
T
the resulting matrix and right-hand side
R
m
= H
(m)
m
= Q
m
H
m
,
g
m
= Q
m
(e
1
) = (
1
, ...,
m+1
)
T
.
Denote by R
m
the mxm upper triangular matrix obtained from R
m
by
deleting its last row and by g
m
the mdimensional vector obtained
from g
m
by deleting its last component.
Then,
1. Vector y
m
which minimizes
e
1
H
m
y
2
is given by
y
m
= R
1
m
g
m
.
2. The residual vector at step m satises
b Ax
m
= V
m+1
(e
1
H
m
y) = V
m+1
Q
T
m
(
m+1
e
m+1
) and, as a result,
kb Ax
m
k
2
= j
m+1
j .
This was the process for computing the least-squares solution y
m
. The
process must be stopped if the residual norm j
m+1
j is small enough.
The last rows of R
m
and g
m
are deleted and the resulting upper
triangular system is solved to obtain y
m
. Then the approximate
solution x
m
= x
0
+V
m
y
m
is computed.
The Symmetric Lanczos Algorithm
This algorithm can be viewed as a simplication of Arnoldis method
for the particular case when the matrix is symmetric. When A is
symmetric, then the Hessenberg matrix H
m
becomes symmetric
tridiagonal.
0
B
B
B
B
@
1

2
2

2

3
...
m1

m1

m
m

m
1
C
C
C
C
A
The Conjugate Gradient Algorithm
This algorithm is one of the best known iterative techniques for
solving sparse Symmetric Positive Denite linear systems.
Assume we need to minimize the following function
f (x) =
1
2
x
T
Ax x
T
b
where A is nxn-matrix positive denite and symmetric, b is n-vector.
The minimum value of f (x) is b
T
A
1
b/2, achieved by setting
x = A
1
b. Therefore, minimizing f (x) and solving Ax = b are
equivalent problems if A is symmetric positive denite.
The vector x
j +1
can be expressed as .
x
j +1
= x
j
+
j
p
j
.
Therefore, the residual vectors must satisfy the recurrence
r
j +1
= r
j

j
Ap
j
.
To have r
j
s orthogonal it is necessary that
(r
j

j
Ap
j
, r
j
) = 0
and, as a result,
j
=
(r
j ,
r
j
)
(Ap
j
, r
j
)
.
The rst basis vector p
1
is the gradient of f at x
0
, which equals to
Ax
0
b . The other vectors in the basis will be conjugate to the
gradient . Each next p
k+1
is dened to be in the direction closest to
the gradient r
k
under the conjugacy constraint. This direction is given
by the projection of r
k
onto the space orthogonal to p
k
with respect
to the inner product induced by A.
p
j +1
= r
j +1
+
j
p
j
,
(Ap
j
, r
j
) = (Ap
j
, p
j

j 1
p
j 1
) = (Ap
j
, p
j
)
j
=
(r
j ,
r
j
)
(Ap
j
, p
j
)
,
j
=
(r
j +1,
Ap
j
)
(p
j
, Ap
j
)
Ap
j
=
1
j
(r
j +1
r
j
),
j
=
1
j
(r
j +1,
(r
j +1
r
j
))
(Ap
j
, p
j
)
=
(r
j +1
, r
j +1
)
(r
j
, r
j
)
The process stops if r
j +1
is "suciently small".
Convergence Analysis
One of the main tool used in the analysis of convergence behavior is
Chebyshev polynomials.
Lemma 1. Let x
m
be the approximate solution obtained from the
mth step of the CG algorithm, and let d
m
= x
x
m
, where x
is
the exact solution. Then x
m
is of the form
x
m
= x
0
+q
m
(A)r
0
,
where q
m
is a polynomial of degree m 1 such that
k(I Aq
m
(A))d
0
k
A
= min
q2P
m1
k(I Aq(A))d
0
k
A
.
Theorem 2. Let x
m
be the approximate solution obtained from the
mth step of the CG algorithm, and x
is the exact solution. Then

kx
x
m
k
A
2
"
p
k 1
p
k + 1
#
m
kx
x
0
k
A
.

Krylov 2 Filimon

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Krylov 2 Filimon

Transféré par

Droits d'auteur :

Formats disponibles

Krylov Subspace Iteration Methods

is the exact solution. Then

Vous aimerez peut-être aussi