Académique Documents
Professionnel Documents
Culture Documents
Lecture 6.
Set of linear equations
OUTLINE
Exact methods
Nave Gaussian Elimination
Gauss-Siedel Method
LU Decomposition
a m1 x1 a m 2 x2 a mn xn bm
Ax b
possible solutions:
Kronecker-Capelli theorem
a11 a1n
A
a m1 a mn
a11 a1n b1
C
a m1 a mn bm
Kronecker-Capelli theorem
Kronecker-Capelli theorem
System of m linear equations with n unknowns has
a solution if the rank r of the main matrix equals to
the rank of the expanded matrix :
rank A = rank C = r
Kronecker-Capelli theorem
Kronecker-Capelli theorem
If the rank r of both matrices is equal to the number
of unknowns, there is one solution of the system of
equations , i.e., one set of numbers satisfying the
equation can be found; a system is consistent
rank A = rank C = n
Kronecker-Capelli theorem
Kronecker-Capelli theorem
If common order r the both matrix is smaller than
the number of unknowns n, then (n - r) unknowns
can be chosen arbitrarily, and the remaining r
unknowns determined from the equation, then the
system has no unique solution, it depends on (n - r)
parameters
Kronecker-Capelli theorem
Kronecker-Capelli theorem
if the rank r of the main matrix is smaller than the
rank of an extended matrix, the system of linear
equations has no solution; the system is
inconsistent
rank A rank C
In the space Rn, whose elements are the vectors: x x1 , x2, ..., xn
x 1 x1 x2 ... xn
x 2 x x ... x
2
1
2
2
2 1/ 2
n
max x1 , x2 , ..., xn
x 2 x1 n x 2 nx
10
11
Exact methods
Exact methods - features
A small number of calculations needed to determine the
solution
If the task is ill-conditioned numerically, the solution can
be subject to a significant error.
They can be unstable due to rounding off errors
The transformation matrix A uses to a large extent the
machine's memory, especially if the original data A and b
should be kept to a final verification
12
a11 x1 a12 x2 b1
a21 x1 a22 x2 b2
a22b1 a12b2
x1
a11a22 a21a12
Method 1:
a11b2 a21b1
x2
a11a22 a21a12
13
x1 0,80
x2 0,36
14
We receive:
a11
0,99
0,7070 0,71
0,00 x2 0,00
15
i 1
i 1
17
.....................................................
Generally
xi
ann xn bn
bn
xn
ann
i n 1, n 2, ..., 1
18
1 2 1
M n n
2
2
1 2 1
D n n
2
2
19
Gaussian Elimination
The first phase (the phase of the elimination of the
"forward coefficient)
.....................
an1 x1 an 2 x2 an 3 x3 ... ann xn bn
n-1 steps of elimination are required
20
Gaussian Elimination
Step 1. From the second row subtracting the first one, divided by
a11 and multiplied by a21
a21
a11
a21
a21
a21
a21 x1
a12 x2 ...
a1n xn
b1
a11
a11
a11
a21
a21
a21
a22
a12 x2 ... a2 n
a1n xn b2
b1
a11
a11
a11
21
Gaussian Elimination
'
'
a22
x2 a23
x3 ... a2' n xn b2'
'
'
a32
x2 a33
x3 ... a3' n xn b3'
.....................
'
an' 2 x2 an' 3 x3 ... ann
xn bn'
'
where: a22
a22
.
.
.
a2' n a2 n
Numerical Methods - Lecture 6
a21
a12
a11
a21
a1n
a11
22
Gaussian Elimination
Step 2. We repeat the procedure step 1 for the third
row
a '32
'
'
'
'
'
a32
x2
'
a23
a'22
'
'
'
a32
a32
' a32
'
x
...
a
x
b
3
2n '
n
2
'
'
a22
a22
a22
'
'
a32
x2 a33
x3 ... a3' n xn b3'
We receive :
'
a '32
a '32
a '32
a33
a '23 x3 ... a '3n
a '2 n xn b'3
b '2
a '22
a '22
a '22
23
Gaussian Elimination
'
'
a22
x2 a23
x3 ... a2' n xn b2'
"
a33
x3 ... a3" n xn b3"
.....................
a x ... a x b
"
n3 3
"
nn n
"
n
24
Gaussian Elimination
At the end of step n-1 the system of equations takes
the form:
"
a33
x3 ... a3" n xn b3"
.....................
n 1
ann xn bn
n 1
25
Gaussian Elimination
After the n-1 step elimination of variables resulting
equations can be written in matrix form:
a11 a12
0 a'
22
0
0
0
0
a1n x1 b1
a'2 n x2 b2'
"
"
a3n x3 b3
(n 1 )
xn bn(n-1 )
0 ann
a13
a'23
a"33
26
Gaussian Elimination
Start with the last equation because it has only one unknown
bn( n 1)
x n ( n 1)
a nn
xi
xi
Numerical Methods - Lecture 6
aii
i 1
bi
i 1
aiji 1 x j
n
j i 1
i 1
ii
for i n 1,...,1
for i n 1,...,1
27
Gaussian Elimination
Gaussian Elimination computational cost
1 3
1
2
M n n n
3
3
28
Example:
Velocity
(m/s)
12
106.8
177.2
279.2
vt a1t 2 a 2 t a3 ,
5 t 12.
29
v t a 1 t 2 a 2 t a 3 , 5 t 12.
t 12
2
t2
t 32
t1
t2
t3
t1 5 s , v ( 5 ) 106 , 8 m / s
t 2 8 s , v ( 5 ) 177 , 2 m / s
t 3 12 s , v ( 5 ) 279 , 2 m / s
25
64
144
5
8
12
1
1
a1
v1
a v
2
2
a 3
v 3
1
1
1
a 1 106 . 8
a 177 . 2
2
a 3 279 . 2
30
144 12 1 279.2
25
Divide equation 1 by
25 and multiply by 64
64
2.56
25
We receive :
Numerical Methods - Lecture 6
64
64
0
12.8
2.56
4.8 1.56
177.2
273.408
96.208
5
1
106.8
25
0 4.8 1.56 96.208
144 12
1
279.2
31
25
144
144
0
144
5.76
25
1
5.76
16.8 4.76
279.2
615.168
335.968
5
1
106.8
25
0 4.8 1.56 96.208
32
106.8
25
0 4.8 1.56 96.208
Divide equation 2
by -4.8 and multiply
by -16.8
16.8
3 .5
4 .8
0
0
0
16.8 4.76
335.968
0.7
0.76
5
1
106.8
25
0 4.8 1.56 96.208
0
0
0.7
0.76
33
0
0
0.7 0.7 0
0
0.7 a3 0.76
Back Substitution
Solving for a3
0.7 a3 0.76
0.76
0 .7
a3 1.08571
a3
34
Solving for a2
5
1 a1 106.8
25
0 4.8 1.56 a 96.208
0
0
0.7 a3 0.76
96.208 1.56a3
a3 1.08571
a2
4.8
96.208 1.56 1.08571
a2
4.8
a2 19.6905
35
0
0
0.7 a3 0.76
Solving for
a1
a3 1.08571
a2 19.6905
106.8 5a2 a3
a1
25
106.8 5 19.6905 1.08571
25
0.290472
36
Solution:
25
64
144
5
8
12
1
1
a1 106 .8
a 177 .2
2
a 3 279 .2
a1 0.290472
a 19.6905
2
a3 1.08571
37
Gaussian Elimination
38
Gaussian Elimination
Division by zero can occur any time during elimination
of variables
12 10 7 x1 15
6 5
3 x2 14
24 1 5 x3 28
12 10 7 x1 15
0
x 6.5
0
6
.
5
2
0 21 19 x3 2
39
Gaussian Elimination
15
10 x1 45
20
3 2.249 7 x 1.751
1
3 x3 9
5
Exact solution
x1
x
2
x3
1
1
1
x1 0.9625
x 1.05
2
x3 0.999995
x1 0.625
x 1.5
2
x3 0.99995
40
Gaussian Elimination
Partial pivoting method
- with a partial choice of the base element
akk
41
Gaussian Elimination
Example :
25 5 1 a1 106.8
64 8 1 a 177.2
144 12 1 a3 279.2
25 , 64 , 144
25 5 1 106.8
144 12 1 279.2
64 8 1 177.2
64 8 1 177.2
144 12 1 279.2
25 5 1 106.8
Exchange the third row and the first row
42
Gaussian Elimination
25 5 1
A 64 8 1
144 12 1
5
1
25
B 0 4.8 1.56
0
0
0.7
43
Gaussian Elimination
Following application of the
method with partial selection of
basic element , we have
received matrix[C]
12
1
144
C 0 2.917 0.8264
0
0
0.2
44
Gaussian Elimination
144 12 1 279.2
64 8 1 177.2
25 5 1 106.8
144
64
0.4444
144
64
63.99
0
1 177.2
5.333 0.4444
124.1
12
1
279.2
144
0 2.667 0.5556 53.10
25
5
1
106.8
45
Gaussian Elimination
12
1
279.2
144
0 2.667 0.5556 53.10
25
5
1
106.8
144
Divide equation 1 by
144 and multiply by
25
25
0.1736
144
25
25
0
1 106.8
2.917
0.8264 58.33
12
1
279.2
144
0 2.667 0.5556 53.10
Gaussian Elimination
The values in the second column of the second and
third row is:
2.667 , 2.917
Maximum is 2.917 in the third row
Replace the third row of the second
12
1
279.2
12
1
279.2
144
144
0 2.667 0.5556 53.10 0 2.917 0.8264 58.33
47
Gaussian Elimination
12
1
279.2 Divide equation 2 by
144
0 2.917 0.8264 58.33 2.917 and multiply by
2.667
0 2.667 0.5556 53.10
2.917
2.667
0.9143.
2.917
0
0
0
2.667 0.5556
2.667 0.7556
0
53.10
53.33
0.2 0.23
12
1
279.2
144
0 2.917 0.8264 58.33
0
0
0.2 0.23
48
Gaussian Elimination
1
144 12
0 2.917 0.8264
0
0.2
0
Solving for a2
a1 279.2
a 58.33
2
a3 0.23
58.33 0.8264a3
a2
2.917
58.33 0.8264 1.15
2.917
19.67
49
Gaussian Elimination
1
144 12
0 2.917 0.8264
0
0.2
0
Solving for a1
a1 279.2
a 58.33
2
a3 0.23
279.2 12a2 a3
a1
144
279.2 12 19.67 1.15
144
0.2917
50
Solution:
25
64
144
Gaussian Elimination
5
8
12
1
1
a 1 106 .8
a 177 .2
2
a 3 279 .2
a1 0.2917
a 19.67
2
a3 1.15
51
Gauss-Seidel Method
System of n equations with n unknowns:
.
.
.
.
.
.
52
Gauss-Seidel Method
from equation 1
from equation 2
with n-1
from equation n
53
Gauss-Seidel Method
General form of the i - th equation
bi aij x j
n
xi
j 1
j i
aii
, i 1,2,, n.
54
Gauss-Seidel Method
We assume the initial values of x1 to xn and substitute
them into previously transformed equations
Calculate the Absolute Relative Approximate Error
a i
xinew xiold
100
new
xi
x1
x
2
xn -1
xn
55
Example:
Gauss-Seidel Method
Time t
(s)
5
Velocity
(m/s)
12
106.8
177.2
279.2
vt a1t 2 a 2 t a3 ,
5 t 12.
56
Gauss-Seidel Method
Using a Matrix
template of the
form:
The system of
equations
becomes:
Initial Guess: Assume an
initial guess of
Numerical Methods - Lecture 6
t12
2
t 2
t32
t1
t2
t3
1 a1 v1
1 a2 v2
1 a3 v3
25 5 1 a1 106 .8
64 8 1 a 177 .2
144 12 1 a3 279 .2
a1 1
a 2
2
a3 5
57
Gauss-Seidel Method
Rewriting each equation:
25 5 1 a1 106 .8
64 8 1 a 177 .2
144 12 1 a3 279 .2
106.8 5a 2 a 3
a1
25
177.2 64a1 a 3
a2
8
279.2 144 a1 12a 2
a3
1
58
Gauss-Seidel Method
The first iteration:
a1 1
a 2
2
a3 5
106.8 5( 2) (5)
a1
3.6720
25
177.2 643.6720 5
a2
7.8510
8
59
Gauss-Seidel Method
xinew xiold
100
new
xi
a 1
3.6720 1.0000
100 72.76%
3.6720
a 2
7.8510 2.0000
100 125.47%
7.8510
a 3
155.36 5.0000
100 103.22%
155.36
a 3 155 .36
The maximum
absolute relative
approximate error
is 125.47%
60
Gauss-Seidel Method
Second iteration:
a1 3.6720
a 7.8510
2
a3 155.36
61
Gauss-Seidel Method
Finding the absolute relative approximate error:
12.056 3.6720
a 1
x100 69.543%
12.056
54.882 7.8510
x100 85.695%
54.882
a1 12.056
a 54.882
2
a3 798.54
The maximum
absolute
798.34 155.36
relative
a 3
x100 80.540%
798.34
approximate
error is
85.695%
62
Gauss-Seidel Method
Iteration
1
2
3
4
5
6
a1
3.6720
12.056
47.182
193.33
800.53
3322.6
a 1 %
72.767
69.543
74.447
75.595
75.850
75.906
a2
7.8510
54.882
255.51
1093.4
4577.2
19049
a 2 %
125.47
85.695
78.521
76.632
76.112
75.972
a3
155.36
798.34
3448.9
14440
60072
24958
0
a 3 %
103.22
80.540
76.852
76.116
75.963
75.931
a 1 0.29048
a 19.690
2
a 3 1.0857
63
Gauss-Seidel Method
If the matrix is strongly diagonally dominant the
Gauss-Seidel method is convergent
aii aij
n
j 1, j i
aii aij
n
j 1, j i
for all i
64
Gauss-Seidel Method
Example of a matrix diagonally dominant
12 3 5
1 5 3
3 7 13
65
LU Decomposition
LU Decomposition is another method to solve a set of
simultaneous linear equations
Ax b
A LU
where:
L lower triangular matrix
U upper triangular matrix
Numerical Methods - Lecture 6
66
LU Decomposition
If solving a set of linear
equations:
Multiply by:
but:
L L I
unit matrix
L U X C
A LU
AX C
L LU X L C
1
I U X L C
1
but:
therefore:
I U U
1
U X L C
L1 C Z
LZ C
U X Z
67
LU Decomposition
U X L C
1
U X Z L1 C Z
LZ C
Numerical Methods - Lecture 6
68
LU Decomposition
Given:
AX C
69
LU Decomposition
[A] Decompose to [L] and [U]
1
A LU 21
31
32
0 u11
0 0
1 0
u12
u 22
0
u13
u 23
u 33
70
LU Decomposition - example
To find the shape of the object with images of the surface in three
dimensions, for example, you need to solve the following system
of equations:
0,2425
0,2357
0
0,2425
0,2357
0,9701 x 1 247
0,9701 x 2 248
0,9428 x 3 239
On the right are the intensity of light from the center of the image.
The matrix coefficients depend on the direction of the light source
relative to the camera. Unknowns are the intensity of the image,
which will determine the shape of the object. We find values x1,
x2, x3 using LU decomposition.
Numerical Methods - Lecture 6
71
Solution:
LU Decomposition example
1
A LU 21
31
0
1
32
0 u11
0 0
1 0
u12
u 22
0
u13
u 23
u 33
72
LU Decomposition - example
First step:
0
0,9701
0,2425
row1
0
row2
(
0
)
0
,
2425
0
,
9701
0,2425
0,2357 0,2357 0,9428
0
0,9701
0,2425
row1
0
row3
0
,
2357
)
0
,
2425
0
,
9701
0,2425
0
0,2357 1,8857
73
LU Decomposition - example
Second step:
0
0,9701
0,2425
row2
0
row3
0
,
2357
)
0
,
2425
0
,
9701
0,2425
0
0
2,8286
0
0,9701
0,2425
U 0 0,2425 0,9701
0
0
2,8286
Numerical Methods - Lecture 6
we determine
matrix [L]:
0
1
[ L] 21 1
31 32
0
0
1
74
LU Decomposition - example
Finding the L matrix:
1
21
31
32
0
0 the first step of
1 finding the matrix U
a 21
0
21
0
a11 0,2425
a 31 0,2357
31
0,97196
a 11
0,2425
32
a 32 0,2357
0,97196
a 22
0,2425
75
LU Decomposition - example
When matrix [L] and [U] are known, we will try to solve
the system [L][Z]=[C]:
1
0,97196
0
1
0,97196
0 z1 247
0 z 2 248
1 z 3 239
z1 247
z 2 248
76
LU Decomposition - example
z1 247
[ Z ] z 2 248
z 3 720 ,12
0,2425
0
0
Numerical Methods - Lecture 6
0
0,2425
0
0,9701 x 1 247
0,9701 x 2 248
2,8286 x 3 720 ,12
77
LU Decomposition - example
we get vector x:
x 1 0,10905
x 4,2328
2
x 3 254 ,59
78
LU Decomposition - example
Homework:
Solve the system of describing the three-phase AC
circuit:
79