Hilbert Spaces

Definition. A complex inner product space (or pre-Hilbert space) is a complex vector
space X together with an inner product: a function from X X into C (denoted by hy, xi)
satisfying:

(1) ( x X) hx, xi 0 and hx, xi = 0 iff x = 0.

(2) ( , C) ( x, y, z X), hz, x + yi = hz, xi + hz, yi.

(3) ( x, y X) hy, xi = hx, yi

Remarks.

(2) says the inner product is linear in the second variable;

(3) says the inner product is sesquilinear;

(2) and (3) imply hx + y, zi = hx, zi, so the inner product is conjugate linear
zi + hy,
in the first variable.

p
Definition. For x X, let kxk = hx, xi.
Cauchy-Schwarz Inequality. ( x, y X) |hy, xi| kxk kyk, with equality iff x and y
are linearly dependent.
Proof. The result is obvious if hy, xi = 0. Suppose hy, xi = 6 0. Then x 6= 0, y 6= 0.
Let z = ||1 y. Then hz, xi = ||1 hy, xi = || > 0. Let v = xkxk1 , w = zkzk1 . Then
kvk = kwk = 1 and hw, vi > 0. Since 0 kv wk2 = hv, vi 2Rehw, vi + hw, wi, it follows
that hw, vi 1 (with equality iff v = w, which happens iff x and y are linearly dependent).
So |hy, xi| = hz, xi = kxk kzkhw, vi kxk kzk = kxk kyk. 

Facts.

(1 ) ( x X)kxk 0; kxk = 0 iff x = 0.

(2 ) ( C)( x X) kxk = || kxk.

(3 ) ( x, y X) kx + yk kxk + kyk.

83
84 Hilbert Spaces

Proof of (3):

kx + yk2 = kxk2 + 2Rehy, xi + kyk2 kxk2 + 2|hy, xi| + kyk2 kxk2 + 2kxk kyk + kyk2.

Hence k k is a norm on X; called the norm induced by the inner product h, i.

Definition. An inner product space which is complete with respect to the norm induced by
the inner product is called a Hilbert space.
Pn
Example. X = Cn . For x = (x1 , . . . , xn ) and y = (y1 , . . . , yn ) Cn , let hy, xi = j=1 yj xj .
qP
n 2 2 n
Then kxk = j=1 |xj | is the l -norm on C .

Examples of Hilbert spaces:

any finite dimensional inner product space

P P
l2 = {(x1 , x2 , x3 , . . .) : xk C, 2
k=1 |xk | < } with hy, xi = k=1 yk xk
R
L2 (A) for any measurable A Rn , with inner product hg, f i = A g(x)f (x)dx.

Incomplete inner product space

Rb
C([a, b]) with hg, f i = a g(x)f (x)dx

C([a, b]) with this inner product is not complete; it is dense in L2 ([a, b]), which is
complete.

Parallelogram Law. Let X be an inner product space. Then ( x, y X)

kx + yk2 + kx yk2 = 2(kxk2 + kyk2).

Proof. kx + yk2 + kx yk2 = hx + y, x + yi + hx y, x yi = hx, xi + hx, yi + hy, xi +

hy, yi + hx, xi hx, yi hy, xi + hy, yi = 2(hx, xi + hy, yi) = 2(kxk2 + kyk2). 

Polarization Identity. Let X be an inner product space. Then ( x, y X)

1

hy, xi = ky + xk2 ky xk2 iky + ixk2 + iky ixk2 .
4
Proof. Expanding out the implied inner products, one shows easily that
ky + xk2 ky xk2 = 4Rehy, xi and ky + ixk2 ky ixk2 = 4hy, xi. 
Note: In a real inner product space, hy, xi = 41 (kx + yk2 kx yk2 ).
Remark. In an inner product space, the inner product determines the norm. The polarization
identity shows that the norm determines the inner product. But not every norm on a vector
space X is induced by an inner product.
Hilbert Spaces 85

Theorem. Suppose (X, k k) is a normed linear space. The norm k k is induced by an

inner product iff the parallelogram law holds in (X, k k).

Proof Sketch. (): see above. (): Use the polarization identity to define h, i. Then
immediately hx, xi = kxk2 , hy, xi = hx, yi, and hy, ixi = ihy, xi. Use the parallelogram law
to show hz, x + yi = hz, xi + hz, yi. Then show hy, xi = hy, xi successively for N,
1 N, Q, R, and finally C.  

Continuity of the Inner Product. Let X be an inner product space with induced norm
k k. Then h, i : X X C is continuous.
Proof. Since X X and C are metric spaces, it suffices to show sequential continuity.
Suppose xn x and yn y. Then by the Schwarz inequality,

|hyn , xn i hy, xi| = |hyn , xn xi + hyn y, xi| kxn xk kyn k + kxk kyn yk 0. 

Orthogonality. If hy, xi = 0, we say x and y are orthogonal and write x y. For any
subset A X, define A = {x X : hy, xi = 0 y A}. Since the inner product is linear
in the second component and continuous, A is a closed subspace of X. Also
(span(A)) = A , A = A , and (span(A)) = A .
The Pythagorean Theorem. If x1 , . . . , xn X and xj xk for j 6= k, then
n
X 2 X n

xj = kxj k2 .

j=1 j=1

Proof. If x y then kx + yk2 = kxk2 + 2Rehy, xi + kyk2 = kxk2 + kyk2. Now use induction.


Convex Sets. A subset A of a vector space X is called convex if ( x, y A) ( t (0, 1))

(1 t)x + ty A.

Examples.
(1) Every subspace is convex.
(2) In a normed linear space, B(x, ) is convex for > 0 and x X.
(3) If A is convex and x X, then A + x {y + x : y A} is convex.

Theorem. Every nonempty closed convex subset A of a Hilbert space X has a unique
element of smallest norm.

Proof. Let = inf{kxk : x A}. If x, y A, then 21 (x + y) A by convexity, and by the

parallelogram law,
kx yk2 = 2(kxk2 + kyk2) kx + yk2 2(kxk2 + kyk2) 4 2 .
86 Hilbert Spaces

Uniqueness follows: if kxk = kyk = , then kxyk2 4 2 4 2 = 0, so x = y. For existence,

choose {yn }
n=1 A for which kyn k . As n, m ,

kyn ym k2 2(kyn k2 + kym k2 ) 4 2 0,

so {yn } is Cauchy. By completeness, y X for which yn y, and since A is closed, y A.

Also kyk = lim kyn k = . 

Corollary. If A is a nonempty closed convex set in a Hilbert space and x X, then a

unique closest element of A to x.

Proof. Let z be the unique smallest element of the nonempty closed convex set A x =
{y x : y A}, and let y = z + x. Then y A is clearly the unique closest element of A to
x. 

Orthogonal Projections onto Closed Subspaces

The Projection Theorem. Let M be a closed subspace of a Hilbert space X.

(1) For each x X, unique u M, v M such that x = u + v. (So as vector spaces,

X = M M .)

Define the operators P : X M and Q : X M by P : x 7 u and Q : x 7 v.

(2) If x M, P x = x and Qx = 0; if x M , P x = 0 and Qx = x.

(3) P 2 = P , Range(P ) = M, Null Space(P ) = M ; Q2 = Q, Range(Q) = M , Null

Space(Q) = M.

(4) P, Q B(X, X). kP k = 0 if M = {0}; otherwise kP k = 1. kQk = 0 if M = {0};

otherwise kQk = 1.

(5) P x is the unique closest element of M to x, and Qx is the unique closest element of
M to x.

(6) P + Q = I (obvious by the definition of P and Q).

Proof Sketch. Given x X, x + M is a closed convex set. Define Qx to be the smallest

element of x + M, and let P x = x Qx. Since Qx x + M, P x M. Let z = Qx.
Suppose y M and kyk = 1, and let = hy, zi. Then z y x + M, so kzk2
kz yk2 = kzk2 hz, yi hy, zi + ||2 = kzk2 ||2. So = 0. Thus z M . Since
clearly M M = {0}, the uniqueness of u and v in (1) follows. (2) is immediate from the
definition. (3) follows from (1) and (2). For x, y X, x+y = (P x+P y)+(Qx+Qy),
so by uniqueness in (1), P (x + y) = P x + P y and Q(x + y) = Qx + Qy. By the
Pythagorean Theorem, kxk2 = kP xk2 + kQxk2 , so P, Q B(X, X) and kP k, kQk 1. The
rest of (4) follows from (2). Fix x X. If y X, then kx yk2 = kP x P yk2 + kQx Qyk2 .
Hilbert Spaces 87

If y M, then kx yk2 = kP x yk2 + kQxk2 , which is clearly minimized by taking y = P x.

If y M , then kxyk2 = kP xk2 +kQxyk2 , which is clearly minimized by taking y = Qx.


Corollary. If M is a closed subspace of a Hilbert space X, then (M ) = M. In general,

for any A X, (A ) = span{A}, which is the smallest closed subspace of X containing A,
often called the closed linear span of A.

Bounded Linear Functionals and Riesz Representation Theorem

Proposition. Let X be an inner product space, fix y X, and define fy : X C by
fy (x) = hy, xi. Then fy X and kfy k = kyk.

Proof. |fy (x)| = |hy, xi| kxk kyk, so fy X and kfy k kyk. Since |fy (y)| = |hy, yi| =
kyk2, kfy k kyk. So kfy k = kyk. 

Theorem. Let X be a Hilbert space.

(1) If f X , then a unique y X f = fy , i.e., f (x) = hy, xi x X.

(2) The map : X X given by : y 7 fy is a conjugate linear isometry of X onto

X .

Proof.

(1) If f 0, let y = 0. If f X and f 6 0, then M f 1 ({0}) is a proper closed

subspace of X, so z M kzk = 1. Let = f (z) and y = z. Given x X,
u f (x)z f (z)x M, so 0 = hz, ui = f (x)hz, zi f (z)hz, xi = f (x) hz, xi =
f (x) hy, xi, i.e., f (x) = hy, xi. Uniqueness: if hy1 , xi = hy2 , xi x X, then (letting
x = y1 y2 ) ky1 y2 k2 = 0, so y1 = y2 .

(2) follows immediately from (1), the previous proposition, and the conjugate linearity of
the inner product in the first variable. 

Corollary. X is a Hilbert space with the inner product hg, f i = h 1 (g), 1(f )i (i.e.,
hfy , fx i = hy, xi = hx, yi).

Proof. Clearly hf, f i 0, hf, f i = 0 iff 1 (f ) = 0 iff f = 0, and hg, f i = hf, gi.
Also hfy , 1 fx1 + 2 fx2 i = hfy , f1 x1 +2 x2 i = hy,
1x1 +
2 x2 i = 1 hy, x1 i + 2 hy, x2i =

1 hfy , fx1 i + 2 hfy , fx2 i, so h, i is an inner product on X . Since hfy , fy i = hy, yi = kyk2 =
kfy k2 , h, i induces the norm on X . Since X is complete, it is a Hilbert space. 

Remark. Part (1) of the Theorem above is often called [one of] the Riesz Representation
Theorem[s].
88 Hilbert Spaces

Strong convergence/Weak convergence

Let X be a Hilbert space. We say xn x strongly if kxn xk 0 as n . This
is the usual concept of convergence in the metric induced by the norm, and is also called
convergence in norm. We say xn x weakly if ( y X) hy, xn i hy, xi as n .
w
(Other common notations for weak convergence are xn x, xn x.) The Cauchy-Schwarz
inequality shows that strong convergence implies weak convergence. Also, if xn x strongly,
then kxn k kxk since |kxn k kxk| kxn xk.
Example. (Weak convergence does not imply strong convergence if dim X = ). Let
k th entry
2
X = l . For k = 1, 2, . . ., let ek = (0, . . . , 0, 1, 0, . . .) (so {ek : k = 1, 2, . . .} is an
2
orthonormal set in l ).

Claim. ek 0 weakly as k .
P
Proof. Fix y l2 . Then k=1 |yk |
2
< , so yk 0. So hy, ek i = yk 0. 
Note that kek k = 1, so ek does not converge to zero strongly.
Remark. If dim X < , then weak convergence strong convergence (exercise).

Theorem. Suppose xn x weakly in a Hilbert space X. Then

(a) kxk lim inf k kxk k

(b) If kxk k kxk, then xk x strongly (i.e., kxk xk 0).

Proof.

(a) 0 kx xk k2 = kxk2 2Rehx, xk i + kxk k2 . By hypothesis, hx, xk i hx, xi = kxk2 .

So taking lim inf above, 0 kxk2 2kxk2 + lim inf kxk k2 , i.e. kxk2 lim inf kxk k2 .

(b) If xk x weakly and kxk k kxk, then kx xk k2 = kxk2 2Rehxk , xi + kxk k2

kxk2 2kxk2 + kxk2 = 0. 

Remark. The Uniform Boundedness Principle implies that if xk x weakly, then kxk k is
bounded.

Orthogonal Sets

Definition. Let X be an inner product space. Let A be a set (not necessarily countable).
A set {u}A X is called an orthogonal set if ( 6= A) hu , ui = 0. (Often it is
also assumed that each u 6= 0.)
Hilbert Spaces 89

Orthonormal Sets
Definition. Let X be an inner product space. A set {u }A is called an orthonormal set
if it is orthogonal and ( A) ku k = 1. For each x X, define a function x b:AC
by x b() = hu , xi. The x
b()s are called the Fourier coefficients of x with respect to the
orthonormal set {u}A .

Theorem.
Pk If {u1 , . . . , uk } is an orthonormal set in an Pl inner 2product space X, and x =
2
c u
j=1 j j , then c j = hu j , xi for 1 j k and kxk = j=1 |cj |
P
Proof. hui , xi = cj hui , uj i = ci . Now use the Pythagorean Theorem. 

Corollary. Every orthonormal set is linearly independent.

Example. If A is finite, say A = {1,P 2, . . . , n}, then for any x X, we know that the closest
element of span{u1 , . . . , un } to x is nk=1 huk , xiuk .

Theorem. (Gram-Schmidt process) Let V be a subspace of an inner product space X,

and suppose V has a finite or countable basis {xn }n1 . Then V has a basis {un }n1 which
is orthonormal (we reserve the term orthonormal basis to mean something else); moreover
we can choose {un }n1 so that for all m 1, span{u1, . . . , um } = span{x1 , . . . , xm }.
x1
Proof Sketch.. Define {un } inductively. Start with u1 = kx1 k
. Having defined u1 , . . . , un1,
Pn1
let vn = xn j=1 huj , xn iuj and un = kvvnn k . 

Theorem. Let V be a finite dimensional subspace of a Hilbert space X. Let {u1, . . . , un }

be a basis forPV which is orthonormal, and let P be the orthogonal
P projection of X onto V .
Then P x = nj=1 huj , xiuj and kxk2 = kP xk2 + kQxk2 = nj=1 |huj , xi|2 + kQxk2 .

Definition.
P Let A be aP
nonempty set. For each A, let y be a nonnegative real number.
Define A y = sup{ F y : F A and F is finite}.
P
Proposition. If A y < , then y 6= 0 for at most countably many .

Proof. For each k, it is clear that Ak { : y > k 1 } is a finite set. But { : y 6= 0} =

k=1 Ak . 
2
Definition.P Let A be 2a nonempty set.2 Define l (A) to be the set of functions f : A C
for which A |f ()| < . Then l (A) is a Hilbert space with inner product hg, f i =
P p
A g()f () and norm kf k2 = hf, f i.

Bessels Inequality. Let {uP }A be an orthonormal set in a Hilbert space X, let x X,

and let x x()|2 kxk2 .
b() = hu , xi. Then A |b

Proof. By the previous Theorem, this is true for every finite subset of A. Take the sup. 

Corollary. Let {u}A , x be as above. Then

90 Hilbert Spaces

b l2 (A) and kb
(1) x xk2 kxk

(2) { A : x
b() 6= 0} is countable.

Theorem. Let X be a Hilbert space and let {u }A be an orthonormal set. Define

F : X l2 (A) (F is for Fourier) by F (x) = x
b where x
b() = hu, xi. Then F is a bounded
linear operator with kF k = 1, which maps X onto l2 (A).

Proof. Clearly F is linear. By (1) of the Corollary, F is bounded and kF k 1. If x = u for

some A, kb l2 (A), f () 6= 0 only for a countable
xk2 = 1 = kxk, so kF k = 1. Given f P
set Af A; enumerate them 1 , 2 , 3 , . . .. Let xk = kj=1 f (j )uj . Clearly xbk () = f ()
for 1 , . . . , k and x
bk () = 0 otherwise. So

X
kb
xk f k22 = |f (j )|2 0 as k .
j=k+1

Thus x bk f in l2 (A), and in particular x

bk is a Cauchy sequence in l2 (A). Since each xk is
a finite linear combination of the u s, kxj xk k = kb
xj x
bk k2 , so {xk } is Cauchy in X, so
xk x in X for some x X. For each A,

b() = hu , xi = lim hu, xk i = lim x

x bk () = f ().
k k

So F (x) = f and F is onto. 

Theorem. Let X be a Hilbert space. Every orthonormal set in X is contained in a maximal

orthonormal set (i.e., an orthonormal set not properly contained in any orthonormal set).

Proof. Zorns lemma. 

Corollary. Every Hilbert space has a maximal orthonormal set.

Theorem. Let {u }A be an orthonormal set in a Hilbert space X. The following condi-

tions are equivalent:
(a) {u}A is a maximal orthonormal set.

(b) The set of finite linear combinations of the u s is dense in X.

P
(c) ( x X) kxk2 = A |b x()|2 (Parsevals relation).
P
(d) ( x, y X) hy, xi = A yb()b x().

(e) ( x X) if ( A) hu , xi = 0 then x = 0.

Proof.
(a) (b): Let V = span{u : A} and M = V . Then M is a closed subspace. Since
{u} is maximal, V = {0}, so M = {0}, so M = X.
Hilbert Spaces 91

(b) (c): Clear if x = 0. Given x 6= 0, and given > 0 (WLOG assume < kxk), choose
y V kx yk < , say y span{u1 , . . . , uk }. Let z = x
b(1 )u1 + + x
b(k )uk .
Then z minimizes kx wk over w span{u1 , . . . , uk } so kx zk kx yk < .
P P
Thus kxk < kzk + , so (kxk )2 < kzk2 and kzk2 = kj=1 |b x(j )|2 A |b x()|2 .
P
Let 0 to get kxk2 A |b x()|2. The other inequality is Bessels inequality.
(c) (d): Use polarization. Only countably many terms in the sum are nonzero.
b() 0, so kxk2 = hx, xi = 0, so x = 0.
(d) (e): Suppose ( A) hx, u i = 0. Then x
(e) (a): If {u } is not maximal, then x 6= 0 hx, u i = 0 for all A. 

Definition. An orthonormal set {u } in a Hilbert space X satisfying the conditions in the

previous theorem is called a complete orthonormal set (or a complete orthonormal system)
or an orthonormal basis in X.
Caution. If X is infinite dimensional, an orthonormal basis is not a basis in the usual
definition of a basis for a vector space (i.e., each x X has a unique representation as a
finite linear combination of basis elements). Such a basis in this context is called a Hamel
basis.
Definition. Let X and Y be inner product spaces. A map T : X Y which is linear,
bijective, and preserves inner products (i.e., ( x, y X) hx, yi = hT x, T yi this implies T
is an isometry kxk = kT xk) is called a unitary isomorphism.
Corollary. If X is a Hilbert space and {u }A is an orthonormal basis of X, then the map
F : X l2 (A) mapping x 7 x b (where x
b() = hx, ui) is a unitary isomorphism.
Corollary. Every Hilbert space is unitarily isomorphic to l2 (A) for some A.

Norm Convergence of Fourier Series

Theorem. Let X be a Hilbert space, {u }A be an orthonormal set inPX, and let x X.
Let {j }j1 be any enumeration of { A : hu, xi =6 0}. Then kxk2 = j1 |huj , xi|2 (i.e.
P

Parsevals Equality holds for this x) iff limn x nj=1huj , xiuj = 0 (i.e. the Fourier
P
series b(j )uj converges to x in norm).
j=1 x

Proof. Let Mn = span{u1 , . . . , un } and let Pn be the orthogonal

P projection onto Mn (so
I Pn is the orthogonal projection onto Mn ). Then Pn x = nj=1 huj , xiuj and kPn xk2 =
Pn 2 2 2 2 2 2 2
j=1 |huPj , xi| . Also kxk = kPn xk + k(I Pn )xk , so kxk kPn xk = kx Pn xk . Hence
kxk2 = j1 |huj , xi|2 iff limn kPn xk2 = kxk2 iff limn kx Pn xk2 = 0, which is the
desired conclusion. (Note: If { A : hu , xi P
=
6 0} is finite, say {1 , . . . , n }, then Parseval
holds iff kPn xk2 = kxk2 iff x = Pn x, i.e., x = nj=1huj , xiuj Mn .)
Corollary. Let {u}A be an orthonormal set in a Hilbert space X. Then {u } is an
orthonormal basis
P iff for each x X and each enumeration {j }j1 of { A : hu, xi =
6 0},
limn kx nj=1huj , xiuj k = 0.
92 Hilbert Spaces

Cardinality of Orthonormal Bases

Proposition. l2 (A) is unitarily isomorphic to l2 (B) iff card(A) = card(B).
Proposition. Any pair of orthonormal bases in a Hilbert space have the same cardinality.
Proposition. A Hilbert space X is separable iff it has a countable orthonormal basis.
Remark. For a separable Hilbert space X, one can show directly without invoking Zorns
lemma that X has a countable complete orthonormal set.
Proof. Clear if dim X < . Suppose dim X = . Let z1 , z2 , . . . be a countable dense
subset. Apply Gram-Schmidt (dropping zero vectors along the way) to get an orthonormal
sequence u1 , u2 , . . . whose finite linear combinations include z1 , z, . . ., and thus are dense. 
Theorem. (Orthogonal projection in terms of orthonormal bases.) Let X be a Hilbert
space, and let M be a closed subspace of X. Let {v }B be a complete orthonormal set in
M, and let {w }C be a complete orthonormal set in M . Then {v } P
{w } is a complete
orthonormal set in X. The orthogonal projection of X onto M is P x = B hv , xiv , and
P
the orthogonal projection of X onto M is Qx = C hw , xiw .

Proof. Follows directly from X = M M and the projection theorem. 

Example. (Orthogonal Polynomials in weighted L2 spaces.) Fix a, b R with < a <
Rb
b < . Let w C(a, b) with w(x) > 0 on (a, b) and a w(x)dx < . The function w is
called the weight function. For example, take w(x) = (1 x2 )1/2 on (1, 1). Define
 Z b 
2 2
Lw (a, b) = f : f is measurable on (a, b) and |f (x)| w(x)dx <
a
Rb
and define hg, f iw = a g(x)f (x)w(x)dx for f, g L2w (a, b). Then (after identifying f and g
when f = g a.e.), L2w (a, b) is a Hilbert space.
Claim. Polynomials are dense in L2w (a, b).
Rb
Proof. First note that if f L (a, b), then f L2w (a, b) since a |f (x)|2 w(x)dx
Rb R  21
b
kf k2 a w(x)dx, and thus kf kw Mkf k , where M = a
w(x)dx < . Given
f L2w (a, b), g C[a, b] for which kf gkw < 12 (exercise). By the Weierstrass Approx-
imation Theorem, polynomials are dense in (C[a, b], k k ), so a polynomial p for which
kg pk < (2M)1 . Then kf pkw kf gkw + kg pkw < 12 + Mkg pk < 12 + 12 = .

Applying the Gram-Schmidt process to the linearly independent set {1, x, x2 , . . .} using the
inner product of L2w (a, b) produces a sequence of polynomials which are orthonormal in
L2w (a, b).
Theorem. The orthogonal polynomials in L2w (a, b) are a complete orthonormal set in
L2w (a, b).
Proof. Finite linear combinations are dense.