Académique Documents
Professionnel Documents
Culture Documents
LECTURE FOUR
Solving linear systems with special properties
1. Understand possible scenarios where systems of linear equations that contains multiple
right-hand-side are relevant and learn how to solve these systems
2. Define, explain and determine systems that are either symmetric, positive definite, tri-
diagonal and diagonally dominant.
3. Understand systems with special properties and learn the specific methods for solving
these systems with more efficient methods.
These systems occur very often in practical applications, such as in numerical solution of partial
differential equations. There are hardly any systems in practical applications that are not one of
the above types. Hence, these systems deserve special treatment.
1
Lecture Notes-Numerical Analysis math352-Solving Special systems of linear equations
Solve:
5
Ux1 Mb1 0.2857
0
0.3333
x1 0.3333
0
Solve:
6
Ux2 Mb2 1.1429
0
0.6667
x2 1.3333
0
The above can be used to find the inverse of matrix.
2
Lecture Notes-Numerical Analysis math352-Solving Special systems of linear equations
Because the matrix A is the same for all the systems, A has to be factored only once. Thus, if
Gaussian elimination with partial pivoting is used, then bi is replaced by ei in the case of
multiple RHS.
ALGORITHM
Example 2.4.3
1 2 4
A 4 5 6
7 8 9
Step 1:
MA U where
7 8 9 0 0 1
U 0 7 19 1 0 71
7 , M
6
1
1 1 1
0 0 2 2 2
Step 2:
x3 2.6667 3.3333 1
T
1 4.6667 2.6667
Thus, A X 2
1
6.3333 3.3333
1 2.000 1
3
Lecture Notes-Numerical Analysis math352-Solving Special systems of linear equations
MA U
Then, det M det A det U . Now, M M n1Pn1...M 2 P2 M1P1 . Because the determinant of
each of the lower elementary matrices is 1 and the determinant of each of the permutation
matrices 1 , we have
det M 1
r
Where r is the number of row interchanges made during the pivoting process. So we have
1
det A det U
det M
1 u11 u22
r
unn
1 a11a22
1 n 1
r
ann
4
Lecture Notes-Numerical Analysis math352-Solving Special systems of linear equations
det A 1 a11a22
1 n 1
r
ann where r is the number of interchanges.
Let r and s be, respectively, the number of row and column interchanges. Then
det M 1
r
det Q 1
s
det A 1 det U
rs
1 1 n 1
r s
a11a22 ann
EXAMPLE
0 1 1
A 1 2 3
1 1 1
1 2 4
U 0 1 1
0 0 1
3 2 1
U 0 23 13
1
0 0 2
5
Lecture Notes-Numerical Analysis math352-Solving Special systems of linear equations
In the first step, one row interchange and one column interchange occurred. In the second step,
one row interchange and one column interchange occurred. Thus, r 2, s 2 , and
det A 1 det U 1 3 23 12 1
r s 4
This factorization is called the Cholesky factorization (decomposition). The existence of the
Cholesky factorization for a symmetric positive definite matrix A can be seen either via LU
factorization of A or by finding the matrix H directly from the preceding relation. In practical
computations, the latter is preferable.
We have
ai1
h11 a11 , hi1 , i 2,..., n
h11
i j
h
k 1
2
ik aii , aij hik h jk j i
k 1
6
Lecture Notes-Numerical Analysis math352-Solving Special systems of linear equations
Algorithm
for k 1, 2, , n do
for i 1, 2, , k 1 do
1 i 1
hki ki hij hkj
a
hii j 1
end
k 1
hkk akk hkj2
j 1
end
Remarks:
0
1. In the preceding pseudocode, ( ) 0 . Also when k = 1, the inner loop
j1
is skipped.
2. The Cholesky factor H is computed row by row.
3. The positive definiteness of A will make the quantities under the square root sign
positive.
Procedure:
The Cholesky Algorithm for the Positive definite System Ax b .
7
Lecture Notes-Numerical Analysis math352-Solving Special systems of linear equations
Example
1 1 1 3
Let A 1 5 5 , b 11
1 5 14 20
Step 2: Solve H y b
1 0 0 y1 3
1 2 0 y2 11
1 2 3 y 20
3
y1 3, y2 4, y3 3
Step 3: Solve H T x y :
1 1 1 x1 3
0 2 2 x2 4
0 0 3 x 3
3
x3 1, x2 1, x3 1
8
Lecture Notes-Numerical Analysis math352-Solving Special systems of linear equations
5. Tri-diagonal Systems
At the end of this section you will be able to solve linear system problems Ax b where A has
special characteristics. The study includes Gaussian elimination for special systems: Tri-
diagonal, and Block Tri-diagonal Systems.
The LU factorization of a tri-diagonal matrix T, when it exists, may yield L and U having very
special simple structures: both bi-diagonal, L having 1s along the main diagonal and the
superdiagonal entries of U the same as those of T. In particular, if we write
1 u1 b1
a1 b1
c l2 0
0
T 2
bn 1
an
0 0 bn 1
1 un
cn
ln
By equating the corresponding elements of the matrices on both sides, we see that
a1 u1
ci li ui 1 , i 2, ,n
ai ui li bi 1 , i 2, ,n
from which li and ui can be easily computed.
Set u1 a1
a1 b1 For i 2, , n do
c
Let T 2
bn 1
, c
li i
cn an ui 1
ui ai li bi 1
FIop count.
The solutions of these two bidiagonal systems also require 2n 2 flops. Thus, a tridiagonal
system can be solved by this procedure in only 4n-4 flops, a very cheap procedure indeed.
9
Lecture Notes-Numerical Analysis math352-Solving Special systems of linear equations
Example
Triangularize
0.9 0.1 0
A 0.8 0.5 0.1 using (a) the formula A = LU and (b) Gaussian elimination.
0 0.1 0.5
(a) From A LU ,
u1 0.9
i 2:
c 0.8 8
l2 2 0.8889
u1 0.9 9
8
u2 a2 l2b1 0.5 0.1 0.4111
9
i 3:
c 0.1
l3 2 0.2432
u2 0.41
u3 a3 l3b2 0.5 0.24 0.1 0.4757
1 0 0
L 0.8889 1 0
0 0.2432 0.1
0.9 0.1 0
U 0 0.4111 0.1
0 0 0.4757
(b) Using Gaussian elimination with partial pivoting, we have the fol1owing
0.8
Step 1: Multiplier m21 0.8889 :
0.9
0.9 0.1 0
1
A 0 0.4111 0.1
0 0.5
0.1
0.1
Step 2: Multiplier m32 0.2432
0.4111
10
Lecture Notes-Numerical Analysis math352-Solving Special systems of linear equations
0.9 0.1 0
2
A 0 0.4111 0.1 U
0 0.4757
0
1 0 0 1 0 0
L m21 1 0 0.8889 1 0
0 m32 1 0 0.2432 1
11