Math 415 - Lecture 18

Inner Product and Orthogonality

Friday October 7th 2016

Textbook reading: Ch 3.1
Suggested practice exercises: 1, 2, 4, 5, 14, 16
Khan Academy video: Vector Dot Product and Vector Length
Strang lectures: Lecture 30: Linear Transformations / Lecture 14: Orthogonality
Applications: Information retrieval, Support Vector Machine (SVM) in Machine Learning

Review
A linear map T : V W satisfies
T (cx + dy) = cT (x) + dT (y).
T : Rn Rm defined by T (x) = Ax is linear. A is the matrix representing
T in the standard bases. For example, T (e1 ) = Ae1 = first column of A.

e1 = ..

.
0
Any T : V W can be represented by a matrix.
What is the Point? Why write T : V W as a matrix?
Replace obscure computations in V and W by transparent computations
with matrices.
Even if T : Rn Rm (already have standard coordinates), T may be
simpler in a different coordinate system.
1

Summary: Given v in V , want to calculate T (v) in W . Take an input

basis A = (x1 , x2 , . . . , xn ) and and output basis B = (y1 , y2 , . . . , ym ).
The abstract input vector v and the coordinate vector vA determine each
other.
The abstract output vector T (v) and the coordinate vector T (v)B determine each other.
So we know T if we know the matrix TBA :
T (v)B = TBA vA .
The output coordinate vector equals the matrix for T times the input coordinate
vector.
Formula For the Coordinate matrix. To write T : V W as a matrix, take an input basis A = (x1 , x2 , . . . , xn ) and and output basis B =
(y1 , y2 , . . . , ym ). Then


TBA = T (x1 )B T (x2 )B . . . T (xn )B
Example 1. Let T : R3 R3 be reflection across the x-yplane,

(x,y,z) 7
0
0
1
(x, y, z). Determine the matrix representing T in the basis 1 , 1 , 0 .

1
0
1
T : (x, y, z) 7 (x, y, z). So calculate



1
1
0
0
1
T 1 = 1 = 1 + 0 1 2 0
1
1
0
1
1



0
0
0
0
T 1 = 1 , T 0 = (1) 0
1
1
0
0
Hence

0 0
1 0 .
0 1

1
A= 0
2

Example 2. Let T : P3 P2 be the linear map given by

T (p(t)) =

d
p(t).
dt

Whats the matrix A representing T in the standard bases?





Solution. The standard bases are 1, t, t2 , t3 for P2 and 1, t, t2 for P2 . The
matrix A has 4 columns and 3 rows.
2


0
T (1) = 0, so the first column is 0.
0

1
T (t) = 1, so the second column is 0.
0

0
T (t2 ) = 2t, so the third column is 2.
0

0
T (t3 ) = 3t2 , so the last (fourth) column is 0.
3
So the matrix A representing T in the

0 1
0 0
0 0

standard bases is

0 0
2 0 .
0 3

0
What is Col(A) and N ul(A) for A = 0
0

1
0
0

0
2
0

0
0?
3

Solution. Col(A) = R3 . Every quadratic polynomial is the derivative of some

cubic polynomial.

1

0
N ul(A) = span
0 .

0
The corresponding polynomial is p(t) = 1. That makes sense because differentiation kills constant polynomials.
Lets try differentiating 7t3 t + 3 using the matrix A.

3

0 1 0 0
1
1
0 0 2 0 = 0
0
0 0 0 3
21
7
We get 1 + 0t + 21t2 , which is indeed the derivative of 7t3 t + 3.

Inner Product and Distances

Definition. The inner product (or dot product) of v, w Rn is

v w = v T w = v1 w 1 + + vn w n .
Example 3.

1
1

2 1 = 1
3
2

 1
3 1 = 1 2 6 = 7
2

Theorem 1. Let u, v and w be vectors in Rn , and let c be any scalar. Then

(a) u v = v u
(b) (u + v) w = u w + v w
(c) (cu) v =c (u v) = u (cv)
(d) u u 0, and u u = 0 if and only if u = 0.
The norm (or length) of a vector v Rn is
q

kvk = v v = v12 + + vn2 .

Definition.

The distance between points v, w Rn is

dist (v, w) = kv wk.
Example 4.

(a)

1

p

1 = 12 + (1)2 + 32 = 11


3

(b)

dist

  


p
x1
x
x x2
, 2
=k 1
k = (x1 x2 )2 + (y1 y2 )2
y1
y2
y1 y2

Inner Product and Angles

We can use the dot product to compute angles too.

Theorem 2. If v and w are linearly independent, they form an angle , and
v w = kvkkwk cos

v w
v

Example 5. What is the angle formed in R3 between the vectors

0
1
v = 1 , w = 1?
1
0
(A base jumper runs at a cliff at a 45 angle, then jumps straight away from
the cliff and 45 downwards; what angle does he turn as he jumps?)
Solution.

1
0
v = 1 , w = 1
0
1
p

kvk = 12 + 12 + 02 = 2
p

kwk = 02 + (1)2 + (1)2 = 2

v w = 1
v w = kvkkwk cos
1 = 2 cos

cos =

1
2

= 120

Orthogonal vectors

Definition. v and w in Rn are orthogonal if

v w = 0.
Remark. We write v w when v and w are orthogonal. Nonzero vectors v,
w are orthogonal if and only if they are perpendicular.
Nonzero vectors v, w are orthogonal if and only if they are perpendicular.
We can derive this from Pythagoras theorem. v w kvk2 + kwk2 =
kv wk2 v v + w w = (v w) (v w) v v + w w =
v v 2v w + w w v w = 0
5

v w
v
w

Example 6. Are the following vectors orthogonal?

   
1
2
(a)
,
2
1
   
1
2

= 1 (2) + 2 1 = 0
2
1
So yes, theyre orthogonal.

1
2
(b) 2 , 1
1
1

2
1
2 1 = 1 (2) + 2 1 + 1 1 = 1
1
1
So no, theyre not orthogonal.

2
Example 7. Let v = 1. Is the set of vectors orthogonal to v a subspace of
1
R3 ?
V = {x R3 : v x = 0}
Solution.
vx=0

1 1 x = 0

V is just the null space of the matrix vT = 2 1

2


1 . So yes, it is a subspace.

Definition. If V is a subspace of Rn , a vector x is orthogonal to V if it is

orthogonal to every vector in V .
 
 
1
1
Example 8. Let V = span
. Is x =
orthogonal to V ?
1
1
Solution.

   
1
a

= a + a = 0
1
a

So yes.