Derivation of Matrix Inversion Lemma

1 Block Matrix Inversion

i am trying to nd
_
A
rr
B
rs
C
sr
D
ss
_
1
, so ill call it
_
E
rr
F
rs
G
sr
H
ss
_
, & try to calculate it. the obvious
_
A
rr
B
rs
C
sr
D
ss
__
E
rr
F
rs
G
sr
H
ss
_
=
_
I
rr
0
rs
0
sr
I
ss
_
_
A
rr
B
rs
C
sr
D
ss
__
E
rr
F
rs
G
sr
H
ss
_
=
_
_
_
_
A
rr
E
rr
+
singular if r > s
..
B
rs
G
sr
= I
rr
A
rr
F
rs
+B
rs
H
ss
= 0
rs
C
sr
E
rr
+D
ss
G
sr
= 0
sr
C
sr
F
rs
. .
singular if r < s
+D
ss
H
ss
= I
ss
_
_
_
_
0
sr
G = D
1
CE, so D has to be nonsingular.
Substituting G in I
rr
gets E =
_
ABD
1
C
_
1
. they call T = ABD
1
C Schur complement of D.
0
rs
F = A
1
BH, so A has to be nonsingular.
Substituting F in I
ss
gets H =
_
D CA
1
B
_
1
. they call S = D CA
1
B Schur complement of A.
putting things back into a matrix
_
A
rr
B
rs
C
sr
D
ss
_
1
=
_
_
ABD
1
C
_
1
A
1
BH
D
1
CE
_
D CA
1
B
_
1
_
am i missing something? Yes, of course, this expression shouldnt contain H or E. Substitute for them to
get
_
A
rr
B
rs
C
sr
D
ss
_
1
=
_
_
ABD
1
C
_
1
A
1
B
_
D CA
1
B
_
1
D
1
C
_
ABD
1
C
_
1
_
D CA
1
B
_
1
_
(1)
since the composite block matrix is nite (r +s) (r +s), its product with the inverse will commute, this
leads to
_
E
rr
F
rs
G
sr
H
ss
__
A
rr
B
rs
C
sr
D
ss
_
=
_
I
rr
0
rs
0
sr
I
ss
_
_
E
rr
F
rs
G
sr
H
ss
__
A
rr
B
rs
C
sr
D
ss
_
=
_
_
_
_
E
rr
A
rr
+
singular if r > s
..
F
rs
C
sr
= I
rr
E
rr
B
rs
+F
rs
D
ss
= 0
rs
G
sr
A
rr
+H
ss
C
sr
= 0
sr
G
sr
B
rs
. .
singular if r < s
+H
ss
D
ss
= I
ss
_
_
_
_
0
sr
G = HCA
1
, so A has to be nonsingular.
Substituting G in I
ss
gets H =
_
D CA
1
B
_
1
0
rs
F = EBD
1
, so D has to be nonsingular.
Substituting F in I
rr
gets E =
_
ABD
1
C
_
1
putting things back into a matrix
_
A
rr
B
rs
C
sr
D
ss
_
1
=
_
_
ABD
1
C
_
1

_
ABD
1
C
_
1
BD
1

_
D CA
1
B
_
1
CA
1
_
D CA
1
B
_
1
_
(2)
since (1)=(2) we have
_
ABD
1
C
_
1
BD
1
= A
1
B
_
D CA
1
B
_
1
= F
rs
D
1
C
_
ABD
1
C
_
1
=
_
D CA
1
B
_
1
CA
1
= G
sr
1
2 Block Elimination
_
I 0
CA
1
I
__
A B
C D
_
=
_
A B
0 D CA
1
B
_
_
A B
C D
_ _
I A
1
B
0 I
_
=
_
A 0
C D CA
1
B
_
_
I 0
CA
1
I
__
A B
C D
_ _
I A
1
B
0 I
_
=
_
A 0
0 D CA
1
B
_
inverting both sides
_
I A
1
B
0 I
_
1
_
A B
C D
_
1
_
I 0
CA
1
I
_
1
=
_
A
1
0
0 (D CA
1
B)
1
_
_
A B
C D
_
1
=
_
I A
1
B
0 I
__
A
1
0
0 (D CA
1
B)
1
__
I 0
CA
1
I
_
_
A B
C D
_
1
=
_
A
1
+A
1
B(D CA
1
B)
1
CA
1
A
1
B(D CA
1
B)
1
(D CA
1
B)
1
CA
1
(D CA
1
B)
1
_
. .
if both A and (D CA
1
B) nonsingular
=
_
E F
G H
_
(3)
similarly
_
I BD
1
0 I
__
A B
C D
_
=
_
ABD
1
C 0
C D
_
_
A B
C D
_ _
I 0
D
1
C I
_
=
_
ABD
1
C B
0 D
_
_
I BD
1
0 I
__
A B
C D
_ _
I 0
D
1
C I
_
=
_
ABD
1
C 0
0 D
_
inverting both sides
_
I 0
D
1
C I
_
1
_
A B
C D
_
1
_
I BD
1
0 I
_
1
=
_
(ABD
1
C)
1
0
0 D
1
_
_
A B
C D
_
1
=
_
I 0
D
1
C I
__
(ABD
1
C)
1
0
0 D
1
__
I BD
1
0 I
_
_
A B
C D
_
1
=
_
(ABD
1
C)
1
(ABD
1
C)
1
BD
1
D
1
C(ABD
1
C)
1
D
1
+D
1
C(ABD
1
C)
1
BD
1
_
. .
if both D and (A BD
1
C) nonsingular
=
_
E F
G H
_
(4)
compactly
_
A B
C D
_
1
=
_
A
1
+A
1
BS
1
CA
1
A
1
BS
1
S
1
CA
1
S
1
_
. .
if both A and S nonsingular
=
_
T
1
T
1
BD
1
D
1
CT
1
D
1
+D
1
CT
1
BD
1
_
. .
if both D and T nonsingular
=
_
E F
G H
_
E in (1) = E in (3)
_
ABD
1
C
_
1
= A
1
+A
1
B
_
D CA
1
B
_
1
CA
1
H in (1) = H in (4)
_
D CA
1
B
_
1
= D
1
+D
1
C
_
ABD
1
C
_
1
BD
1
2
3 Old way, to get same results. No insight.
here ill repeat the same process with minor changes
_
A
rr
B
rs
C
sr
D
ss
__
E
rr
F
rs
G
sr
H
ss
_
=
_
_
_
_
A
rr
E
rr
+
singular if r > s
..
B
rs
G
sr
= I
rr
A
rr
F
rs
+B
rs
H
ss
= 0
rs
C
sr
E
rr
+D
ss
G
sr
= 0
sr
C
sr
F
rs
. .
singular if r < s
+D
ss
H
ss
= I
ss
_
_
_
_
I
rr
E = A
1
(I
rr
BG), so A has to be nonsingular.
0
sr
G = D
1
CE, so D has to be nonsingular.
G in E gives us E = A
1
(I
rr
+BD
1
CE) = A
1
+A
1
BD
1
CE = E, then
( A
1
A
. .
was I
rr
A
1
BD
1
C)E = A
1
E = (ABD
1
C)
1
. seems like a complicated way to get same crap.
E in G gives us G = D
1
CA
1
(I
rr
BG) = D
1
CA
1
+D
1
CA
1
BG = G, then
(D
1
D
. .
was I
ss
D
1
CA
1
B)G = D
1
CA
1
G = (D CA
1
B)
1
CA
1
, and once again nothing new.
I
ss
H = D
1
(I
ss
CF), so D has to be nonsingular.
0
rs
F = A
1
BH, so A has to be nonsingular.
F in H gives us H = D
1
(I
ss
+CA
1
BH) = D
1
+D
1
CA
1
BH = H, then
( D
1
D
. .
was I
ss
D
1
CA
1
B)H = D
1
H = (D CA
1
B)
1
. seems like a complicated way to get same crap.
H in F gives us F = A
1
BD
1
(I
ss
CF) = A
1
BD
1
+A
1
BD
1
CF = F, then
( A
1
A
. .
was I
rr
A
1
BD
1
C)F = A
1
BD
1
F = (ABD
1
C)
1
BD
1
, and once again nothing new.
but this gives us (2) directly without considering the commutativity, so ill try the commutative case now
and see whatll happen.
_
E
rr
F
rs
G
sr
H
ss
__
A
rr
B
rs
C
sr
D
ss
_
=
_
_
_
_
E
rr
A
rr
+
singular if r > s
..
F
rs
C
sr
= I
rr
E
rr
B
rs
+F
rs
D
ss
= 0
rs
G
sr
A
rr
+H
ss
C
sr
= 0
sr
G
sr
B
rs
. .
singular if r < s
+H
ss
D
ss
= I
ss
_
_
_
_
I
rr
E = (I
rr
FC)A
1
, so A has to be nonsingular.
0
rs
F = E
1
BD
1
, so D has to be nonsingular.
F in E gives us E = (I
rr
+EBD
1
C)A
1
= A
1
+EBD
1
CA
1
= E, then
E( AA
1
. .
was I
rr
BD
1
CA
1
) = A
1
E = (ABD
1
C)
1
. seems like a complicated way to get same crap.
E in F gives us F = (I
rr
FC)A
1
BD
1
= A
1
BD
1
+FCA
1
BD
1
= F, then
F(DD
1
. .
was I
ss
CA
1
BD
1
)G = A
1
BD
1
F = A
1
B(D CA
1
B)
1
, and once again nothing new.
I
ss
H = (I
ss
GB)D
1
, so D has to be nonsingular.
0
sr
G = HCA
1
, so A has to be nonsingular.
H in G gives us G = (I
ss
GB)D
1
CA
1
= (D
1
CA
1
GBD
1
CA
1
) = G, then
G( AA
1
. .
was I
rr
BD
1
CA
1
) = D
1
CA
1
G = D
1
C(ABD
1
C)
1
.
G in H gives us H = (I
ss
+HCA
1
B)D
1
= D
1
+HCA
1
BD
1
= H, then
H(DD
1
. .
was I
ss
CA
1
BD
1
) = D
1
H = (D CA
1
B)
1
, and once again nothing new.
Surprise Surprise, this gives us (1).
3
4 Symmetric case
if
_
A B
C D
_
is symmetric then C = B
T
& A = A
T
, (A
1
)
T
= A
1
. D = D
T
, (D
1
)
T
= D
1
. F
T
= G.
_
A B
B
T
D
_
1
=
from (1)
..
_
(ABD
1
B
T
)
1
A
1
B(D B
T
A
1
B)
1
D
1
B
T
(ABD
1
B
T
)
1
(D B
T
A
1
B)
1
_
=
from (2)
..
_
(ABD
1
B
T
)
1
(ABD
1
B
T
)
1
BD
1
(D B
T
A
1
B)
1
B
T
A
1
(D B
T
A
1
B)
1
_
=
_
E F
F
T
H
_
Schur complements are symmetric, obviously.
_
ABD
1
B
T
_
1
= A
1
+A
1
B
_
D B
T
A
1
B
_
1
B
T
A
1
_
D B
T
A
1
B
_
1
= D
1
+D
1
B
T
_
ABD
1
B
T
_
1
BD
1
more over like on the bottom of the rst page we have
A
1
B(D B
T
A
1
B)
1
= (ABD
1
B
T
)
1
BD
1
= F
rs
D
1
B
T
(ABD
1
B
T
)
1
= (D B
T
A
1
B)
1
B
T
A
1
= F
T
sr
, transposing this line gets us
A
1
B(D B
T
A
1
B)
1
= (ABD
1
B
T
)
1
BD
1
= F
rs
.
5 What next?
have to show:
_
ABD
1
C
_
1
BD
1
= A
1
B
_
D CA
1
B
_
1
= F
rs
D
1
C
_
ABD
1
C
_
1
=
_
D CA
1
B
_
1
CA
1
= G
sr
trivial exercise. and in symmetric case
A
1
B(D B
T
A
1
B)
1
= (ABD
1
B
T
)
1
BD
1
= F
rs
and why
T
1
= A
1
+A
1
BS
1
CA
1
S
1
= D
1
+D
1
CT
1
BD
1
moreover what can be extracted from these two?
6 Notes
The matrix inversion lemma or Woodbury Identity is also known as Sherman-Morrison Formula, W.J.
Duncan in 1944, L. Guttman in 1946 and M. S. Bartlett in 1951 along with dozens other people
discovered this identity independently. When i was working on RLS adaptive ltering i was using it
without knowing how it was derived, so i couldnt sleep at night till i did. It was a good exercise.
Note that we make no assumptions about any block being positive denite, unlike Haykin claims in
his Adaptive Filtering Theory.
If some blocks were positive denite then Schur Complement theorem holds for positive denite ma-
trices.
my name is Nart Shawash
http://personalwebs.oakland.edu/~nmshawas/
4