Chapter 2

Matrices
2.1 Operations with Matrices
2.2 Properties of Matrix Operations
2.3 The Inverse of a Matrix
2.4 Elementary Matrices
2.5 Markov Chains
2.6 Applications of Matrix Operations
2.7 Homework 1 and 2
※ Since any real number can be expressed as a 11 matrix, all rules, operations, or
theorems for matrices can be applied to real numbers, but not vice versa
※ Just memorize those rules that are different for matrices and real numbers 2.1
 2.1 Operations with Matrices
 Matrix:
 a11 a12 a13  a1n 
a a a  a 
 21 22 23 2n 

A  [aij ]   a31 a32 a33  a3n   M mn

 
    
am1 am 2 am3  amn 
mn

(i, j)-th entry (or element): aij

number of rows: m
number of columns: n
size: m×n

 Square matrix: m = n
2.2
Equal matrices: two matrices are equal if they have the same size
(m × n) and entries corresponding to the same position are equal

For A  [aij ]m n and B  [bij ]m n ,

A  B if and only if aij  bij for 1  i  m, 1  j  n

 Ex: Equality of matrices

1 2 a b 
A  B 
3 4   c d 

If A  B, then a  1, b  2, c  3, and d  4

2.3
 Matrix addition:
If A  [aij ]m n , B  [bij ]m n ,

then A  B  [aij ]mn  [bij ]mn  [aij  bij ]mn  [cij ]mn  C

 Ex 2: Matrix addition

 1 2  1 3  1  1 2  3  0 5
 0 1   1 2   0  1 1  2   1 3
       

 1  1  1  1  0 
  3   3    3  3   0 
       
 2  2  2  2 0

2.4
 Scalar (純量) multiplication:
If A  [aij ]m n and c is a constant scalar,
then cA  [caij ]m n

 Matrix subtraction:
A  B  A  (1) B

 Ex 3: Scalar multiplication and matrix subtraction

 1 2 4  2 0 0
A   3 0  1 B   1  4 3
   
 2 1 2  1 3 2

Find (a) 3A, (b) –B, (c) 3A – B

2.5
Sol:
 1 2 4  31 32 34  3 6 12
(a)
3A  3 3 0  1  3 3 30 3 1   9 0  3
     
 2 1 2  32 31 32  6 3 6
(b)
 2 0 0   2 0 0
 B   1 1  4 3    1 4  3

 
 1 3 2  1  3  2
(c)
 3 6 12  2 0 0  1 6 12
3 A  B   9 0  3   1  4 3   10 4  6
     
 6 3 6  1 3 2  7 0 4

2.6
 Matrix multiplication:
If A  [aij ]m n and B  [bij ]n p ,
then AB  [aij ]m n [bij ]n p  [cij ]m p  C ,
If equal, A and B are multipliable (可乘的)
size of C=AB is m × p
n
where cij   aik bkj  ai1b1 j  ai 2b2 j    ainbnj
k 1

 a11 a12 a1n  c c c1 j c1n 

   b11 b1 j b1n   11 12 
  b b2 j b2 n   
 ai1 ai 2 ain   21   ci1 ci 2 cij cin 
     
 
 
bn1 bnj bnn   
 an1 an 2 ann  cn1 cn 2 cnj cnn 

※ The entry cij is obtained by calculating the sum of the entry-by-entry
product between the i-th row of A and the j-th column of B
2.7
 Ex 4: Find AB
 1 3
   3 2 
A   4 2 B 
  4 1  2 2
 5 0 3 2
Sol:
 (1)(3)  (3)(4) (1)(2)  (3)(1) 
AB  (4)(3)  (2)(4) (4)(2)  (2)(1) 
 (5)(3)  (0)(4) (5)(2)  (0)(1)  3 2
 9 1 
  4 6 
 15 10  3 2
 Note: (1) BA is not multipliable
(2) Even BA is multipliable, it could be that AB ≠ BA 2.8
 Matrix form of a system of linear equations in n variables:
 a11 x1  a12 x2    a1n xn  b1
 a x  a x  a x  b
 21 1 22 2 2n n 2
 m linear equations
 
am1 x1  am 2 x2    amn xn  bm


 a11 a12  a1n   x1   b1 
a a  a x  b 
 21 2n   2  single matrix equation
22
 2
           A xb
     m  n n 1 m 1

 m1
a a m2  a mn   xn  bm 
=

A x b
2.9
 Partitioned matrices (分割矩陣):
row vector (向量)
 a11 a12 a13 a14   r1 
A   a21 a22 a23 a24   r2 
 a31 a32 a33 a34  r3 

 a11 a12 a13 a14  column vector (向量)

A   a21 a22 a23 a24   c1 c2 c3 c4 
 a31 a32 a33 a34 

submatrix
 a11 a12 a13 a14 
 A11 A12  ※ Partitioned matrices can be
A  a21 a22 a23 
a24  
   A21 A22 
used to simplify equations or
to obtain new interpretation of
a31 a32 a33 a34  equations (see the next slide)
2.10
Ax is a linear combination (線性組合) of the column vectors
of matrix A:
 a11 a12 a1n   x1 
a a22 a2 n  x 
A  21
 c1 c2 cn  and x   2
   
   
 am1 am 2 amn   xn 

 a11 x1  a12 x2   a1n xn   a11   a21   a1n 

a x a x   a2 n xn  a  a  a 
 Ax   21 1 22 2  x1    x2      xn  2 n 
21 22

          
       
 am1 x1  am 2 x2   amn xn  m1  m1 
a  m2 
a amn 

=
c1 c2 cn
 x1c1  x2c2   xncn  Ax can be viewed as the linear combination of the
 x1  column vectors of A with coefficients x1, x2,…, xn
x 
 c1 c 2 c n   2  ← You can derive the same result if you perform
  the matrix multiplication for matrix A
  expressed in column vectors and x directly
 xn  2.11
 To practice the exercises in Sec. 2.1, we need to know the
trace operation and the notion of diagonal matrices

 Trace (跡數) operation:

If A  [aij ]n n , then Tr ( A)  a11  a22   ann

 Diagonal matrix (對角矩陣): a square matrix in which

nonzero elements are found only in the principal diagonal

d1 0  0
0 d2  0
A  diag (d1 , d 2 ,, d n )     M nn
   
0  d n 
 0
※ It is the usual notation for a diagonal matrix 2.12
Keywords in Section 2.1:
 equality of matrices: 相等矩陣
 matrix addition: 矩陣相加
 scalar multiplication: 純量積
 matrix multiplication: 矩陣相乘
 partitioned matrix: 分割矩陣
 row vector: 列向量
 column vector: 行向量
 trace: 跡數
 diagonal matrix: 對角矩陣

2.13
 2.2 Properties of Matrix Operations
 Three basic matrix operators, introduced in Sec. 2.1:
(1) matrix addition
(2) scalar multiplication
(3) matrix multiplication
0 0 0
0 0 0 
 Zero matrix (零矩陣): 0mn 
 
 
0 0 0  mn
1 0 0
0 1 0 
 Identity matrix of order n (單位矩陣): I n  
 
 
0 0 1  n n
2.14
 Properties of matrix addition and scalar multiplication:
If A, B, C  M mn , and c, d are scalars,
then (1) A+B = B+A (Commutative property (交換律) of matrix addition)

(2) A+(B+C) = (A+B)+C (Associative property (結合律) of matrix addition)

(3) (cd) A = c (dA) (Associative property of scalar multiplication)

(Multiplicative identity property (乘法單位性質), and 1 is

(4) 1A = A the multiplicative identity (乘法單位元素) for all matrices)
(5) c(A+B) = cA + cB (Distributive (分配律) property of scalar
multiplication over matrix addition)
(6) (c+d) A = cA + dA (Distributive property of scalar
multiplication over real-number addition)

 Notes:
All above properties are very similar to the counterpart
properties for real numbers
2.15
 Properties of zero matrices:
If A  M mn , and c is a scalar,
then (1) A  0mn  A
※ So, 0m×n is also called the additive identity (加法單位元素) for the set of
all m×n matrices
(2) A  ( A)  0mn
※ Thus , –A is called the additive inverse (加法反元素) of A

(3) cA  0mn  c  0 or A  0mn

 Notes:
All above properties are very similar to the counterpart
properties for the real number 0

2.16
 Properties of matrix multiplication:
(1) A(BC) = (AB)C (Associative property of matrix multiplication)
(Distributive property of LHS matrix multiplication
(2) A(B+C) = AB + AC over matrix addition)
(3) (A+B)C = AC + BC (Distributive property of RHS matrix multiplication
over matrix addition)
(4) c(AB) = (cA)B = A(cB) (Associative property of scalar and matrix
multiplication)
※ For real numbers, the properties (2) and (3) are the same since the order for the
multiplication of real numbers is irrelevant
※ The real-number multiplication can satisfy above properties and there is a
commutative property for the real-number multiplication, i.e., cd = dc
 Properties of the identity matrix:
If A  M mn , then (1) AI n  A
(2) I m A  A
※ The role of the real number 1 is similar to the identity matrix. However, 1 is unique
in real numbers and there could be many identity matrices with different sizes 2.17
 Ex 3: Matrix Multiplication is Associative
Calculate (AB)C and A(BC) for
 1 0 
1 2 1 0 2
A  , B  , and C  3 1  .
 2 1 3 2 1   2 4
Sol:
 1 0
 1 2  1 0 2   
( AB)C        3 1 
 2 1 3 2 1  
 2 4 
 1 0 
 5 4 0   17 4
  3 1  
 1 2 3 
 13 14 
 2 4 

2.18
   1 0  
1 2   1 0 2  
A( BC )  
1  3 2 1  
3 1 
2  2 4  

1 2   3 8  17 4 
 
2 1  7 2  13 14 


2.19
 Equipped with the four properties of matrix multiplication,
we can prove a statement on Slide 1.35: if a homogeneous
system has any nontrivial solution, this system must have
infinitely many nontrivial solutions

Suppose there is a nonzero solution x1 for this homegeneous system such

that Ax1  0. Then it is straightforward to show that tx1 must be another
solution, i.e.,
A(tx1 )  t ( Ax1 )  t (0)  0

The fourth property of matrix multiplication on Slide 2.17

Finally, since t can be any real number, it can be concluded that there are
infinitely many solutions for this homogeneous system

2.20
 Definition of Ak : repeated multiplication of a square matrix:
A1  A, A2  AA, , Ak  AA A
k matrices of A

 Properties for Ak:

(1) AjAk = Aj+k
(2) (Aj)k = Ajk
where j and k are nonegative integers and A0 is assumed
to be I

 For diagonal matrices:

 d1 0 0  d1k 0 0
0 d  
0   0 d 2k 0
D 2
D 
k
   
   
0 0 dn   0 0 d nk 
2.21
 Transpose of a matrix (轉置矩陣):

 a11 a12 a1n 

a a22 a2 n 
If A   21
 M mn ,
 
 
 am1 am 2 amn 

 a11 a21 am1 

a a22 am 2 
then AT   12  M n m
 
 
 a1n a2 n amn 
※ The transpose operation is to move the entry aij (original at the position (i, j)) to
the position (j, i)
※ Note that after performing the transpose operation, AT is with the size n×m
2.22
 Ex 8: Find the transpose of the following matrix
 1 2 3 0 1
 2
(a) A    (b) A   4 5 6 
(c) A  2 4

8   
7 8 9  1  1
Sol: (a)  2
A   AT  2 8
8 
(b) 1 2 3 1 4 7
A  4 5 6  AT  2 5 8
   
7 8 9 3 6 9
(c) 0 1
0 1
A  2 4  AT  
2
   1 4  1
 1  1
2.23
 Properties of transposes:
(1) ( AT )T  A
(2) ( A  B)T  AT  BT
(3) (cA)T  c( AT )
(4) ( AB)T  BT AT
※ Properties (2) and (4) can be generalized to the sum or product of
multiple matrices. For example, (A+B+C)T = AT+BT+CT and (ABC)T =
CTBTAT
※ Since a real number also can be viewed as a 1 × 1 matrix, the transpose
of a real number is itself, that is, for a  R , aT = a. In other words,
transpose operation has actually no effect on real numbers

2.24
 Ex 9: Verify that (AB)T and BTAT are equal

 2 1 2  3 1
A   1 0 3 B   2 1
 0 2 1  3 0

Sol:
T
 2 1 2   3 1 
T
 2 1
        2 6 1
( AB)    1 0
T
3  2 1    6 1   
  0 2  1 1 2 
 1 
 3 0 
 
 1 2 

 2 1 0
3 2 3     2 6 1
B A 
T T
  1 0 2   
1 1 0   2 3 1 1 2 
 1

2.25
 Symmetric matrix (對稱矩陣):
A square matrix A is symmetric if A = AT
 Skew-symmetric matrix (反對稱矩陣):
A square matrix A is skew-symmetric if AT = –A

 Ex:
 1 2 3
If A  a 4 5 is symmetric, find a, b, c?
 
b c 6
Sol:
 1 2 3 1 a b 
 T
A  a 4 5 AT  2 4 c 
A A
     a  2, b  3, c  5
b c 6 3 5 6

2.26
 Ex:
 0 1 2
If A  a 0 3 is a skew-symmetric, find a, b, c?
 
b c 0
Sol:
 0 1 2  0  a  b
A   a 0 3  AT    1 0  c 
b c 0  
 2  3 0 

A   AT  a  1, b  2, c  3

 Note: AAT must be symmetric
Pf: ( AAT )T  ( AT )T AT  AAT
 AA is symmetric
T
※ The matrix A could be with any size,
i.e., it is not necessary for A to be a
square matrix.
※ In fact, AAT must be a square matrix.

2.27
 Before finishing this section, two properties will be discussed,
which are held for real numbers, but not for matrices: the first is
the commutative property of matrix multiplication and the second
is the cancellation law
 Real number:
ab = ba (Commutative property of real-number multiplication)

 Matrix:
AB  BA
m n n p n  p m  n

Three results for BA:

(1) If m  p , then AB is defined, but BA is undefined
(2) If m  p, m  n, then AB  M mm , BA  M nn (Sizes are not the same)
(3) If m  p  n, then AB  M mm , BA  M mm
(Sizes are the same, but resulting matrices may not be equal)
2.28
 Ex 4:
Show that AB and BA are not equal for the following matrices.
 1 3 2  1
A  and B 
 2  1   0 2 
Sol:
 1 3 2  1 2 5
AB       
 2  1 0 2   4  4 

2  1  1 3 0 7
BA       
 0 2  2  1  4  2 

AB  BA (noncommutativity of matrix multiplication)

2.29
 Notes:

(1) A+B = B+A (the commutative law of matrix addition)

(2) AB  BA (the matrix multiplication is not with the

commutative law) (so the order of matrix multiplication is very
important)

※ This property is different from the property for the

multiplication operations of real numbers, for which the
order of multiplication makes no difference

2.30
 Real number:
ac  bc and c  0
 a b (Cancellation law for real numbers)

 Matrix:
AC  BC and C  0 (C is not a zero matrix)
(1) If C is invertible, then A = B
(2) If C is not invertible, then A  B (Cancellation law is not
necessary to be valid)

※ Here I skip to introduce the definition of “invertible” (可逆的)

because it will be studied soon in the next section

2.31
 Ex 5: An example in which cancellation is not valid
Show that AC=BC
 1 3  2 4  1  2
A  , B  , C 
 0 1  2 3   1 2 
Sol:
1 3  1  2  2 4
AC     
0 1  1 
2   1 2
2 4  1  2   2 4
BC   
2
  
3  1 2    1 2

So, although AC  BC , A  B

2.32
Keywords in Section 2.2:
 zero matrix: 零矩陣
 identity matrix: 單位矩陣
 commutative property: 交換律
 associative property: 結合律
 distributive property: 分配律
 cancellation law: 消去法則
 transpose matrix: 轉置矩陣
 symmetric matrix: 對稱矩陣
 skew-symmetric matrix: 反對稱矩陣

2.33
 2.3 The Inverse of a Matrix
 Inverse matrix (反矩陣):
Consider A  M nn ,
if there exists a matrix B  M nn such that AB  BA  I n ,
then (1) A is invertible (可逆) (or nonsingular (非奇異))
(2) B is the inverse of A

 Note:
A square matrix that does not have an inverse is called
noninvertible (or singular (奇異))
※ The definition of the inverse of a matrix is similar to that of the inverse of a
scalar, i.e., c · (1/c) = 1
※ Since there is no inverse (or said multiplicative inverse (倒數)) for the real
number 0, you can “imagine” that noninvertible matrices act a similar role to
the real number 0 in some sense
2.34
 Theorem 2.7: The inverse of a matrix is unique
If B and C are both inverses of the matrix A, then B = C.
Pf: AB  I
C ( AB)  CI
(CA) B  C  (associative property of matrix multiplication and the property
for the identity matrix)
IB  C
BC
Consequently, the inverse of a matrix is unique.

 Notes:
(1) The inverse of A is denoted by A1
(2) AA1  A1 A  I

2.35
 Find the inverse of a matrix by the Gauss-Jordan elimination:

A | I  
Gauss-Jordan elimination
  I | A1 

 Ex 2: Find the inverse of the matrix A

 1 4
A 
  1  3
Sol:
AX  I
 1 4  x11 x12   1 0
 1  3  x  
   21 x22  0 1
 x11  4 x21 x12  4 x22   1 0
 x  3 x  
 11 21  x12  3x22  0 1
2.36
 by equating corresponding entries
x11  4 x21  1 This two systems of linear
 (1)
 x11  3 x21  0 equations share the same
coefficient matrix, which
x12  4 x22  0 is exactly the matrix A
(2)
 x12  3 x22  1
 1 4 1 A1,2(1) , A2,1( 4)  1 0 3
(1)        x11  3, x21  1
 1 3 0 0 1 1

 1 4 0 A1,2(1) , A2,1( 4)  1 0 4

(2)        x12  4, x22  1
 1 3 1 0 1 1
Thus
Perform the Gauss-
  3  4
1 Jordan elimination on
X A 
1 1  the matrix A with the
same row operations
2.37
 Note:
Instead of solving the two systems separately, you can
solve them simultaneously by appending (附加) the
identity matrix to the right of the coefficient matrix

 1 4 1 0  Gauss-Jordan elimination  1 0 3 4 
 1 3   ( 4 )  0 1 1
(1)

 0 1 A1,2 , A2,1
 1
A I I A1

x  x 
solution for  11  solution for  12 
 x21   x22 

※ If A cannot be row reduced to I, then A is singular

2.38
 Ex 3: Find the inverse of the following matrix

 1  1 0
A   1 0  1
 6 2 3
Sol:
 1  1 0  1 0 0
 A  I    1 0  1  0 1 0
 6 2 3  0 0 1
 1 1 0 1 0 0  1 1 0 1 0 0
  0 1 1 1 1 0   0 1 1 0 
( 1) ( 6)

1 1
A1,3

A1,2

 6 2 3 0 0 1 0 4 3 6 0 1

 1 1 0 1 0 0  1 1 0 1 0 0
 0 1 1 1 1 0  0 1 1 1 1 0 
( 4) M 3( 1)

A2,3

0 0 1 2 4 1 0 0 1 2 4 1

2.39
  1 1 0 1 0 0 1 0 0 2 3 1
0 1 0 3 3 1  0 1 0 3 3 1
(1) (1)
A2,1

A3,2

0 0 1 2 4 1 0 0 1 1 4 1

 [ I  A1 ]

So the matrix A is invertible, and its inverse is

 2  3  1
A1    3  3  1
 2  4  1

 Check it by yourselves:
AA1  A1 A  I

2.40
 Matrix Operations in Excel
 TRANSPOSE: calculate the transpose of a matrix
 MMULT: matrix multiplication
 MINVERSE: calculate the inverse of a matrix
 MDETERM: calculate the determinant of a matrix
 SUMPRODUCT: calculate the inner product of two vectors
※ For TRANSPOSE, MMULT, and MINVSRSE, since the output should
be a matrix or a vector, we need to input the formula in a cell first, then
identify the output range (the cell with the input formula located at the
position a11), and finally put focus on the formula description cell and
press “Ctrl+Shift+Enter” to obtain the desired result
※ See “Matrix operations in Excel for Ch2.xlsx” downloaded from my
website
2.41
 Theorem 2.8: Properties of inverse matrices
If A is an invertible matrix, k is a positive integer, and c is a scalar,
then
(1) A1 is invertible and ( A1 ) 1  A

(2) Ak is invertible and ( Ak ) 1  ( A1 ) k

1 1
1
(3) cA is invertible given c  0 and (cA)  A
c
(4) AT is invertible and ( AT ) 1  ( A1 )T ← “T” is not the number of
power. It denotes the
transpose operation.

Pf:
1 1
(1) A A  I ; (2) A ( A )  I ; (3) (cA)( A )  I ; (4) AT ( A1 )T  I
1 k 1 k

c
2.42
 Theorem 2.9: The inverse of a product
If A and B are invertible matrices of order n, then AB is invertible
and
( AB) 1  B 1 A1
Pf:
( AB)( B1 A1 )  A( BB 1 ) A1  A( I ) A1  ( AI ) A1  AA1  I
(associative property of matrix multiplication)

Thus, if AB is invertible, then its inverse is B1 A1

 Note:
(1) It can be generalized to the product of multiple matrices
 A1 A2 A3  An   An1  A31 A21 A11
1

(2) It is similar to the results of the transpose of the products of

multiple matrices (see Slide 2.24)
 A1 A2 A3 An   AnT A3T A2T A1T
T
2.43
 Theorem 2.10: Cancellation properties for matrix multiplication
If C is an invertible matrix, then the following properties hold:
(1) If AC=BC, then A=B (right cancellation property)
(2) If CA=CB, then A=B (left cancellation property)
Pf:
AC  BC
( AC )C 1  ( BC )C 1 (C is invertible, so C -1 exists)
A(CC 1 )  B(CC 1 ) (Associative property of matrix multiplication)
AI  BI
AB
 Note:
If C is not invertible, then cancellation is not valid
2.44
 Theorem 2.11: Systems of equations with a unique solution
If A is an invertible matrix, then the system of linear equations
Ax = b has a unique solution given by x  A1b
Pf:
Ax  b
A1 Ax  A1b ( A is nonsingular)

Ix  A1b
x  A1b

If x1 and x2 were two solutions of equation Ax  b,

then Ax1  b  Ax2  x1  x2 (left cancellation property)

So, this solution is unique

2.45
 Ex 8:
Use an inverse matrix to solve each system
(a) (b)
2 x  3 y  z  1 2x  3 y  z  4
3x  3 y  z  1 3x  3 y  z  8
2 x  4 y  z  2 2x  4 y  z  5
(c)
2x  3 y  z  0
3x  3 y  z  0
2x  4 y  z  0
Sol:
 2 3 1  1 1 0 
 A   3 3 1 
Gauss-Jordan elimination
 A1   1 0 1 
 2 4 1  6 2 3
2.46
(a)
 1 1 0   1  2  ※ This technique is very
x  A1b   1 0 1   1    1
convenient when you face
the problem of solving
 6 2 3  2  2 several systems with the
same coefficient matrix
(b)
※ Because once you have A-1,
 1 1 0   4  4  you simply need to
x  A1b   1 0 1  8    1  perform the matrix
multiplication to solve the
 6 2 3 5   7  unknown variables
(c) ※ If you solve only one
system, the computational
 1 1 0  0 0 effort is less for the G. E.
x  A1b   1 0 1  0  0
plus the back substitution
or the G. J. E.
 6 2 3 0 0
2.47
Keywords in Section 2.3:
 inverse matrix: 反矩陣
 invertible: 可逆
 nonsingular: 非奇異
 singular: 奇異

2.48
 2.4 Elementary Matrices
 Elementary matrix (列基本矩陣):
An nn matrix is called an elementary matrix if it can be obtained
from the identity matrix I by a single elementary row operation
 Three row elementary matrices:
(1) Ei , j  I i , j ( I ) (interchange two rows)
(2) Ei( k )  M i( k ) ( I ) (k  0) (multiply a row by a nonzero constant)
(3) Ei(,kj)  Ai(,kj) ( I ) (add a multiple of a row to another row)

 Note:
1. Perform only a single elementary row operation on the identity
matrix
2. Since the identity matrix is a square matrix, elementary matrices
must be square matrices 2.49
 Ex 1: Elementary matrices and nonelementary matrices

1 0 0  1 0 0 
(a) 0 3 0
1 0 0 
(b)  (c) 0 1 0

 0 1 0  0 0 0
0 0 1 
Yes (M 2(3) (I 3 )) No (row multiplication must
No (not square)
be with a nonzero constant)

1 0 0  1 0 0 
(d) 0 0 1  1 0  (f ) 0 2 0 
(e)  
0 1 0  2 1  0 0 1

Yes (I 2,3 (I 3 )) (2) No (use two elementary row

Yes (A (I 2 ))
1,2
operations, M 2(2) and M 3( 1) )
2.50
 Note:
Elementary matrices are useful because they enable you to use
matrix multiplication to perform elementary row operations

 Theorem 2.12: Representing elementary row operations

Let E be the elementary matrix obtained by performing an
elementary row operation on Im. If that same elementary row
operation is performed on an mn matrix A, then the resulting
matrix equals the product EA
(Note that the elementary matrix must be multiplied to
the left side of the matrix A)
※ There are many computer programs implementing the matrix addition and
multiplication, but the elementary row operations are seldom to be implemented
※ If you want to perform the elementary row operations on computers (maybe for G.
E. or G.-J. E.), you can compute the matrix multiplication of the corresponding
elementary matrices and the target matrix A instead 2.51
 Ex 3: Using elementary matrices
Find a sequence of elementary matrices that can be used to
rewrite the matrix A in its row-echelon form
0 1 3 5 1  1 3 0 2
 
A   1 3 0 2  0 1 3 5
( )
( 2)
I1,2 , A1,3 , M3 2

 2 6 2 0 0 0 1 2

Sol:
0 1 0  1 0 0
E1  I1,2 ( I 3 )  1 0 0  ( 2)
E2  A1,3 ( I 3 )   0 1 0 
0 0 1   2 0 1

1
1 0 0
( )  
E3  M 3 2 ( I 3 )  0 1 0
0 0 12 
2.52
 0 1 0   0 1 3 5   1 3 0 2
A1  I1,2 ( A)  E1 A  1 0 0 1 3 0 2  0 1 3 5
0 0 1   2 6 2 0   2 6 2 0
 1 0 0  1 3 0 2  1 3 0 2 
( 2)
A2  A1,3 ( A1 )  E2 A1   0 1 0 0 1 3 5   0 1 3 5
 2 0 1   2 6 2 0  0 0 2 4 
 
1 0 0  1 3 0 2   1 3 0 2 
1
 
A3  M 3 2 ( A2 )  E3 A2  0 1 0  0 1 3 5   0 1 3 5  B
( )

 1  0 0 2 4  0 0 1 2 
0 0 
 2 same row-echelon form
1
( )
( 2)
 B  E3 E2 E1 A or B  M 3
2
( A1,3 ( I1,2 ( A)))

※ This example demonstrates that we can obtain the same effect of

performing elementary row operations by multiplying corresponding
elementary matrices to the left side of the target matrix A
2.53
 Row-equivalent (列等價):
Matrix B is row-equivalent to A if there exists a finite number
of elementary matrices such that
※ This row-equivalent
property is from the row-
B  Ek Ek 1  E2 E1 A equivalent property after
performing a finite
sequence of the
elementary row operations
(see Slide 1.23)

 Theorem 2.13: Elementary matrices are invertible

If E is an elementary matrix, then E 1 exists and E 1 is still an
elementary matrix

2.54
 Ex: Elementary Matrix Inverse Matrix

0 1 0  0 1 0 
E1  1 0 0  I1,2 ( I ) E11  1 0 0  I1,2 ( I )
E11 still corresponds to
I1,2(I)
0 0 1  0 0 1 
The corresponding
element row operations
1
 1 0 0 1 0 0  for E2 is still to add a

E2   0 1 0  A1,3 ( I ) E21  0 1 0  A1,3

multiple of the row 1 to
( 2) (2)
  (I ) the row 3, but the
multiplying scalar is
 2 0 1  2 0 1  from –2 to 2 ⇒ Place 2
at position (3,1)
The corresponding

 1 0 0 1 0 0  element row operations

1
for E3 is still to
E31  0 1 0   M 3(2) ( I )
1
  ( )
E3  0 1 0   M 3 2 ( I ) multiply the row 3 by a
nonzero constant, but
0 0 12  0 0 2 the constant is from 1/2
to 2 ⇒ Place 2 at
position (3,3) 2.55
  Theorem 2.14: A property of invertible matrices
A square matrix A is invertible if and only if it can be written as
the product of elementary matrices (we can further obtain that an
invertible matrix is row-equivalent to the identity matrix)
Pf:
() Assume that A is the product of elementary matrices
1. Every elementary matrix is invertible (Thm. 2.13)
2. The product of invertible matrices is invertible (Thm. 2.9)
Thus A is invertible
( ) If A is invertible, Ax  0 has only one solution (Thm. 2.11), and
this solution must be the trivial solution.
  A 0 G.-J. E.
 I 0  Ek E3 E2 E1 A  I  An invertible matrix
is row-equivalent to
 A  E11 E21E31 Ek1I =E1E2 E3 Ek I the identity matrix

Thus A can be written as the product of elementary matrices

2.56
 Ex 4:
Find a sequence of elementary matrices whose product is
  1  2
A 
 3 8
Sol:
 1 2  M1( 1) 1 2  A1,2( 3) 1 2 
A       
 3 8   3 8   0 2 
1 2  A2,1( 2 ) 1 0 
1
( )

    I
M22
 
0 1  0 1 
1
( )
( 2) ( 3) ( 1)
Therefore E2,1 E
2
2
E1,2 E1 A  I

2.57
 1
( )
Thus A  ( E1( 1) ) 1 ( E1,2
( 3) 1
) (E 2
2 1 ( 2) 1
) ( E2,1 )

 1 0 1 0 1 0 1 2
  3 1 0 2 0 1
 0 1    

 Note: another way to justify using the Gauss-Jordan elimination

to find A–1 on Slide 2.36
First, if A is invertible, according to Thm. 2.14,
then Ek E3 E2 E1 A  I . (Therefore, the G.-J. E. can be performed by applying the
counterpart elementary row operations for E1,…E2, and Ek)
Second, since A1 A  I , A1  Ek E3 E2 E1 according to the cancellation rule.
Third , by expressing the G.-J. E. on [ A I ] through the matrix multiplication with
elementary matrices, we obtain Ek E3 E2 E1[ A I ]  [ Ek E3 E2 E1 A Ek E3 E2 E1I ]
 [ I A 1 ] 2.58
 Theorem 2.15: Equivalent conditions
If A is an nn matrix, then the following statements are equivalent
(1) A is invertible
(2) Ax = b has a unique solution for every n1 column vector b

(3) Ax = 0 has only the trivial solution

(4) A is row-equivalent to In
(5) A can be written as the product of elementary matrices

※ Statements (2) and (3) are from Theorem 2.11, and Statements (4) and (5) are
from Theorem 2.14

2.59
 LU-factorization (or LU-decomposition) (LU分解):
If the nn matrix A can be written as the product of a lower
triangular matrix L and an upper triangular matrix U, then
A  LU is a LU -factorization of A

 a11 0 0
a 3  3 lower triangular matrix (下三角矩陣):
 21 a22 0  all entries above the principal diagonal are
 a31 a32 a33  zero

 a11 a12 a13 

0 a 3  3 upper triangular matrix (上三角矩陣):
 22 a23  all entries below the principal diagonal are
 0 0 a33  zero

2.60
 Note:
If a square matrix A can be row reduced to an upper triangular
matrix U using only the elementary row operations of adding a
(𝑘)
multiple of one row to another row below it (i.e., 𝐴𝑖,𝑗 for 𝑗 >
𝑖 ), then A has an LU-factorization

Ek E2 E1 A  U
(U is similar to a row-echelon form matrix, expcet that the
leading coffieicnets may not be 1)
1 1 1 ※ If only Ai(,kj) for 𝑗 > 𝑖 is applied, then E11 E21 Ek1
 A E E 1 2 E U
k will be lower triangular matrices
 A  LU (L  E11 E21 Ek1 )

2.61
 Ex 5 and 6: LU-factorization
 1 3 0 
1 2  0 
(a) A    (b) A  1 3
1 0   
 2 10 2 
Sol: (a)
1 2 A1,2(-1) 1 2 
A      U
1 0  0 2

( 1)
 E1,2 A U
( 1) 1
 A  ( E1,2 ) U  LU
1 0
( 1) 1
 L  (E 1,2 )  
1 1 

2.62
(b)
 1 3 0  1 3 0 1 3 0 

A  0  ( 2)

1 3 
A1,3 
 0  ( 4)

1 3 
A2,3
 0 1 3   U
 2 10 2 0 4 2 0 0 14
※ Note that we do not perform the G.-J. E. here
(4) ( 2) ※ Instead, only Ai(,kj) for 𝑗 > 𝑖 is used to derive the
 E2,3 E1,3 A  U upper triangular matrix U
※ In addition, the inverse of these elementary
( 2) 1 (4) 1
 A  ( E1,3 ) ( E2,3 ) U  LU matrices should be lower triangular matrices
※ Together with the fact that the product of lower
triangular matrices is still a lower triangular
matrix, we can derive the lower triangular matrix
( 2) 1 (4) 1
 L  ( E1,3 ) ( E2,3 ) L in this way

 1 0 0  1 0 0   1 0 0 
 0 1 0  0 1 0    0 1 0 
 2 0 1  0 4 1   2 4 1 
2.63
 Solving Ax=b with an LU-factorization of A (an important
application of the LU-factorization)

Ax  b If A  LU , then LUx  b
Let y  Ux, then Ly  b

 Two steps to solve Ax=b:

(1) Write y = Ux and solve Ly = b for y

(2) Solve Ux = y for x

2.64
 Ex 7: Solving a linear system using LU-factorization
x1  3 x2  5
x2  3 x3   1
2 x1  10 x2  2 x3   20
Sol:
 1  3 0  1 0 0  1  3 0
A  0 1 3  0 1 0  0 1 3  LU
    
2  10 2 2  4 1 0 0 14
(1) Let y  Ux, and solve Ly  b (solved by the forward substitution)

1 0 0  y1    5  y1  5
0 1 0  y2     1   y2  1
2  4 1  y3   20 y3  20  2 y1  4 y2
 20  2(5)  4(1)  14
2.65
(2) Solve the following system Ux  y (solved by the back substitution)
 1  3 0  x1    5  ※ Similar to the method using A –1 to
0 1 3  x2     1  solve the systems, the LU-
0 0 14  x3   14 factorization is useful when you
need to solve many systems with
the same coefficient matrix. In
So x3  1 such scenario, the LU-factorization
is performed once, and can be used
x2  1  3x3  1  (3)(1)  2 many times.
x1  5  3x2  5  3(2)  1 ※ You can find that the
computational effort for LU-
factorization is almost the same as
Thus, the solution is that for the Gaussian elimination,
so if you need to solve one system
1 of linear equations, just use the
x2 Gaussian elimination plus the back
  substitution or the Gauss-Jordan
 1 elimination directly.

2.66
Keywords in Section 2.4:
 row elementary matrix: 列基本矩陣
 row equivalent: 列等價
 lower triangular matrix: 下三角矩陣
 upper triangular matrix: 上三角矩陣
 LU-factorization: LU分解

2.67
 2.5 Markov Chains
 Stochastic Matrices (隨機矩陣)
{S1, S2, …, Sn} is a finite set of state (狀態集合) of a given
population, based on which a stochastic matrix P is defined as
1. The entry 0  pij  1 represents the probability that a
member of a population will change from j-th state to the i-th
state
2. The sum of the entries in the same column is 1
Form
※ A stochastic matrix must
S1 S2 … Sn be a square matrix
 p11 p12  p1n  S
※ The stochastic matrix is
 p+21 + +
p22  p2 n S2 1

P  To
also known as the matrix
…

     of transition probabilities
 p+n1 p+n 2  p+nn  Sn (移轉機率矩陣)

=1 =1 =1 2.68
 Ex 1: Stochastic and nonstochastic matrices

1 0 0  0.1 0.2 0.3

0 1 0  0.2 0.3 0.4
   
 0 0 1   0.3 0.4 0.5 
stochastic not stochastic

1 1 1 1 1 1
2 3 4 2 4 4
1 3 1 2
 0   0 
4 4 3 3
1 2
0 1 3
0
 4 3   4 4 
stochastic not stochastic

2.69
 Ex 2: A consumer preference model
A B None
0.70 0.15 0.15 A
P  0.20 0.80 0.15 B
  None
 0.10 0.05 0.70 

0.15 A ※ X0 is the initial state

X 0  0.20 B
matrix representing the
  portions of the total
0.65 None population in each state

0.70 0.15 0.15 0.15 0.2325 A ※ X1 represents the

    
X 1  PX  0.20 0.80 0.15 0.20  0.2875  portions of the total
     B population in each
0.10 0.05 0.70 0.65 0.4800 None state after one year
2.70
 Markov chain is a sequence 𝑋𝑛 of state matrices that are related
by the equation 𝑋𝑘+1 = 𝑃𝑋𝑘 , where 𝑃 is the stochastic matrix
0.3028 A
X 3  PX 2  PPX 1  PPPX 0  P 3 X 0  0.3904 B After 3 year
 
0.3068 None
 0.3241 A
X 5  P 5 X 0   0.4381 B After 5 year
 
0.2378 None

0.3329 A
X 10  P10 X 0  0.4715 B After 10 year
 
0.1957 None

 0.3333 A
X   P  X 0  0.4762
  B After ∞ year
0.1905 None
2.71
 Property of a steady state matrix (穩定狀態矩陣) 𝑋ത ≡ 𝑋∞
ത 𝑋ത
𝑃𝑋=
The matrix 𝑋𝑛 eventually reaches a steady state. The limit is the
steady state matrix. Left multiplying the steady state matrix with
the stochastic matrix generates the steady state matrix itself

 Regular stochastic matrix (正規化隨機矩陣)

A stochastic matrix 𝑃 is regular when some power of 𝑃 has only
positive entries

 Unique steady state matrix

When 𝑃 is a regular stochastic matrix, the corresponding Markov
chain approaches a unique state matrix 𝑋ത (對於正規化隨機矩陣，
其穩定狀態矩陣是唯一存在) 2.72
 Ex 4: Regular stochastic matrices
0.70 0.15 0.15
(a) P  0.20 0.80 0.15
 
 0.10 0.05 0.70 
P is regular because P1 has only positive entries

0.50 1.00  0.75 0.5

(b) P  P 
2

0.50 0 0.25 0.5
P is regular because P2 has only positive entries

 1 0 1
 13 
(c) P   3 1 0
 1 0 0
3 
P is not regular because every power of P has two zeros
in its second column 2.73
 Ex 5: Finding a steady state matrix
Find the steady state matrix 𝑋ത of the Markov chain whose
matrix of transition probabilities is the regular matrix

0.70 0.15 0.15

P  0.20 0.80 0.15
 
 0.10 0.05 0.70 
Sol:
 x1 
Letting X   x2  . Then use the matrix equation PX  X to
 
 x3 

0.70 0.15 0.15  x1   x1 

    
obtain 0.20 0.80 0.15  x2    x2 
0.10 0.05 0.70  x3   x3 
2.74
or
0.70 x1  0.15 x2  0.15 x3  x1
0.20 x1  0.80 x2  0.15 x3  x2
0.10 x1  0.05 x2  0.70 x3  x3

Use the above equations and the fact that x1 + x2 + x3 = 1 to

write the system of linear equations below
 0.30 x1  0.15 x2  0.15 x3  0
 x1  0.3333
0.20 x1  0.20 x2  0.15 x3  0
 X   x2   0.4762
0.10 x1  0.05 x2  0.30 x3  0    
x1  x2  x3  1  x3  0.1905

Check: PX  X
2.75
 Absorbing state (吸收狀態):
Consider a Markov chain with n different states {S1, S2, . . . , Sn}.
The i-th state Si is an absorbing state when, in the matrix of
transition probabilities P, pii = 1, i.e., this entry on the main
diagonal of P is 1 and all other entries in the i-th column of P are 0

Absorbing Markov chain (吸收馬可夫鏈):

An absorbing Markov chain has the two properties listed below.
1. The Markov chain has at least one absorbing state
2. It is possible for a member of the population to move from any
nonabsorbing state to an absorbing state in a finite number of
transitions (所有非吸收狀態可在有限多次的轉移中到達某個
吸收狀態)
2.76
 Ex 6: Absorbing and nonabsorbing Markov chains
(a) For the matrix
From
S1 S2 S3
S
0.4 0 0  1
P   0 1 0.5 S2 To
0.6 0 0.5 S
3

1. The second state, represented by the second column, is

absorbing
2. The corresponding Markov chain is also absorbing because it is
possible to move from S1 to S2 in two transitions, and it is
possible to move from S3 to S2 in one transition
2.77
(b) For the matrix

From
S1 S2 S3 S4

0.5 0 0 0  S1
0.5 1 0 0  S2
P  To
0 0 0.4 0.5 S3
 0 0 0.6 0.5 S4

1. The second state is absorbing

2. The corresponding Markov chain is not absorbing because
there is no way to move from state S3 or state S4 to state S2

2.78
 Ex 7: Finding steady state matrices of absorbing Markov chains
0.4 0 0 
(a ) P  0 1 0.5  ※ Note that P is not regular
0.6 0 0.5
Use the matrix equation PX  X
0.4 0 0   x1   x1 
0 1 0.5   x    x 
  2   2 
0.6 0 0.5  x3   x3 
along with the equation x1 + x2 + x3 = 1 to derive
 0.6 x1 0
※ The solution coincides
0.5 x3  0  x1  0 with the second column
  x2   1  of the transition
0.6 x1  0.5 x3  0  x   0
 3   probability matrix P
x1  x2  x3  1 2.79
 0.5 0 0.2 0
0.2 1 0.3 0 
( b) P   ※ Note that P is not regular
0.1 0 0.4 0
 
0.2 0 0.1 1
0.5 0  x1   x1 
0 0.2
0.2 0   x2   x2 
1 0.3
Use the matrix equation PX  X       
0.1 0   x3   x3 
0 0.4
    
0.2 1   x4   x4 
0 0.1
along with the equation x1 + x2 + x3 + x4 = 1 to derive
※ The Markov
 0.5 x1  0.2 x3 0 chain has
 x1  0 
0.2 x1  0.3x3  0 x  1  t 
infinitely many
steady state
 0.6 x3  0  x   0  
2
0.1x1 , where t R matrices
0.2 x1  0.1x3 0  3
   (depending on
 x4   t 
x1  x2  x3  x4 1 different initial
state vectors) 2.80
Keywords in Section 2.5:
 stochastic matrix: 隨機矩陣
 transition probability matrix: 移轉機率矩陣
 Markov chain: 馬可夫鏈
 steady state matrix: 穩定狀態矩陣
 regular stochastic matrix: 正規化隨機矩陣
 absorbing state and Markov chain: 吸收狀態與吸收馬可夫鏈

2.81
 2.6 Applications of Matrix Operations
 Least Squares Regression Analysis (Example 10 in the text book
vs. the linear regression in EXCEL)
 Read the text book or “Applications in Ch2.pdf”
downloaded from my website
 See “Example 10 for regression in Ch 2.xlsx” downloaded
from my website for example

2.82
 2.7 Homework 1 and 2
 Homework 1: Implement the matrix multiplication and the
inverse of a matrix with VBA
 For matrix multiplication, input two matrices and output the
resulting matrix (with one CommandButton)
 For matrix inversion, input a matrix and output the inverse
of the matrix (with another CommandButton)
※ Learning goals:
1. How to handle input (輸入) and output (輸出) with VBA
2. Practice using If-Then-Else and For-Next statements
3. How to write and call functions (or subroutine)
※ Total points for this homework is 10. A program that can generate
correct results can earn 10 points
※ CANNOT use any functions, for example, SUM(), TRANSPOSE(),
SUMPRODUCT(), MMULT(), MINVERSE(), or MDETERM(), etc.,
provided by EXCEL
2.83
 Homework 2: Find betas for three component stocks in S&P 500
(10 points) Solve the least squares regression problem as follows
ri,t – rf,t = αi + βi(rM,t – rf,t) + et
– ri,t and rM,t are total returns of the asset i and the market index on each
trading day t, which include both capital gains and dividend income
 Adjusted closing prices of S&P 500 components can be downloaded from
the finance page of U.S. Yahoo (Use adjusted closing prices (rather than
closing prices) to compute the adjusted (or said total) return)
 S&P 500 total return index (with tick symbol XX:SPXT) can be
downloaded from the market data page of The Wall Street Journal
 Convert daily returns, ri,t and rM,t, into annualized returns, i.e., ri,t ×252 and
rM,t ×252, where 252 approximates the number of trading days per year
– The risk free rate rf,t is approximated by 1-month Treasury yields
 U.S. Department of the Treasury: http://www.treasury.gov/resource-
center/data-chart-center/interest-rates/Pages/TextView.aspx?data=yield
– Today is assumed to be July 1st, 2018 and it is required to employ the
prior two-year historical daily returns to solve αi and βi for three firms
2.84
 Bonus: construct a portfolio rp = w1r1 + w2r2 + w3r3 to replicate
the expected return and beta of the market index in the prior two
years, i.e., solving w1, w2, and w3 in the following system (1 point)
𝐸 𝑟𝑝 = 𝑤1 𝐸 𝑟1 + 𝑤2 𝐸 𝑟2 + 𝑤3 𝐸 𝑟3 = 𝐸(𝑟𝑀 )
൞ 𝛽𝑝 = 𝑤1 𝛽1 + 𝑤2 𝛽2 + 𝑤3 𝛽3 = 𝛽𝑀 (= 1)
𝑤1 + 𝑤2 + 𝑤3 = 1
(Note that the expectations are approximated by the sample averages)
– In-sample test (樣本內測試) (1 point): analyze the time series of rp

and rM to verify that rp is indeed with identical average return and

beta as the market portfolio for the prior two years
※ The in-sample test must be exactly correct, and it demonstrates that we can
calculate the portfolio beta by the simple weighted average approach
– Out-of-sample test (樣本外測試) (1 point): compare the average
returns and betas of rp and rM for the next two months and tell me
what you observe
2.85
 Implement Homework 2 with VBA
– Basic requirement (with one CommandButton):
 Inputs: Time series of ri,t, rM,t, and rf,t of the three firms
 Outputs: The values of αi and βi for three firms
– Bonus (with another CommandButton):
 Inputs: Time series of ri,t, rM,t, and rf,t of the three firms and βi for the three
firms
 Outputs: The weights w1, w2 ,and w3 and the expected returns and β of the
portfolio for the past two years and for the following two months

※ When calculating the estimated coefficients of the least squares regression,

one CAN use TRANSPOSE(), SUMPRODUCT(), MMULT(), MINVERSE(),
MDETERM(), SUM(), AVERAGE(), STDEV(), or CORREL() but
CANNOT use other functions provided by EXCEL
※ The basic requirement is 10 points, and the bonus is worth 3 additional points

2.86