PRINCIPLES OF

To Kristine, Sammy, and Justin,

APPLIED
Still my best friends after all these years.
MATHEMATICS

Transformation
and Approximation
Revised Edition

JAMES P. KEENER
University of Utah
Salt Lake City, Utah

ADVANCED BOOK PROGRAM

~ ew
I I I I

Member of the Perseus Books Group

 Contents

Preface to First Edition xi

Preface to Second Edition xvii

1 Finite Dimensional Vector Spaces 1

1.1 Linear Vector Spaces . . . . . . . . 1
1.2 Spectral Theory for Matrices . . . 9
1.3 Geometrical Significance of Eigenvalues 17
1.4 Fredholm Alternative Theorem . . . . . 24
1.5 Least Squares Solutions-Pseudo Inverses . 25
1.5.1 The Problem of Procrustes . . . . 41
Many of the designations used by manufacturers and sellers to distinguish their 42
products are claimed as trademarks. Where those designations appear in this 1.6 Applications of Eigenvalues and Eigenfunctions
book and Westview Press was aware of a trademark claim, the designations have 1.6.1 Exponentiation of Matrices . . . . . . . 42
been printed in initial capital letters. 1.6.2 The Power Method and Positive Matrices 43
1.6.3 Iteration Methods 45
Library of Congress Catalog Card Number: 99-067545
Further Reading . . . . 47
ISBN: 0-7382-0129-4 Problems for Chapter 1 49

Copyright© 2000 by James P. Keener 59

2 Function Spaces
Westview Press books are available at special discounts for bulk purchases in the 2.1 Complete Vector Spaces . . . . . 59
U.S. by corporations, institutions, and other organizations. For more informa- 2.1.1 Sobolev Spaces . . . . . . 65
tion, please contact the Special Markets Department at the Perseus Books Group, 67
2.2 Approximation in Hilbert Spaces
11 Cambridge Center, Cambridge, MA 02142, or call (617) 252-5298. 67
2.2.1 Fourier Series and Completeness
All rights reserved. No part of this publication may be reproduced, stored in a 2.2.2 Orthogonal Polynomials . . . . 69
retrieval system, or transmitted, in any form or by any means, electric, mechani- 2.2.3 Trigonometric Series . . . . . 73
cal, photocopying, recording, or otherwise, without the prior written permission 76
of the publisher. Printed in the United States of America. 2.2.4 Discrete Fourier Transforms .
2.2.5 Sine Functions 78
Westview Press is a member of the Perseus Books Group 79
2.2.6 Wavelets . . . .
2.2. 7 Finite Elements . 88
First printing, December 1999 Further Reading . . . . 92
Problems for Chapter 2 . . . 93
Find us on the World Wide Web at
http:/ /www.westviewpress.com
v
vi CONTENTS vii
CONTENTS

3 Integral Equations 101 6 Complex Variable Theory 209

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 6.1 Complex Valued Functions . . . . . . . . . . . . . . . . . . . . . 209
3.2 Bounded Linear Operators in Hilbert Space . . . . . . . . . . . . 105 6.2 The Calculus of Complex Functions . . . . . . . . . . . . . . . . 214
3.3 Compact Operators . . . . . . . . . . . . . . . . . . . . . . . . . 111 6.2.1 Differentiation-Analytic Functions . . . . . . . . . . . . . 214
3.4 Spectral Theory for Compact Operators . . . . . . . . . . . . . . 114 6.2.2 Integration . . . . . . . . . . . . . . . . . . . . . . . . . . 217
3.5 Resolvent and Pseudo-Resolvent Kernels . . . . . . . . . . . . . . 118
? 6.2.3 Cauchy Integral Formula . . . . . . . . . . . . . . . . . . 220
3.6 Approximate Solutions . . . . . . . . . . . . . . . . . . . . . . . .
3. 7 Singular Integral Equations . . . . . . . . . . . . . . . . . . . . .
Further R.eading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
121
125
127
II
~
6.2.4 Taylor and Laurent Series . . . . . . . . . . . . . . . . . .
6.3 Fluid Flow and Conformal Mappings . . . . . . . . . . . . . . . .
6.3.1 Laplace's Equation . . . . . . . . . . . . . . . . . . . . . .
6.3.2 Conformal Mappings . . . . . . . . . . . . . . . . . . . . .
224
228
228
236
Problems for Chapter 3 . . . . . . . . . . . . . . . .· . . . . . . . . . . 128
4 Differential Operators
4.1 Distributions and the Delta Function . . . . . . . . . . . . . . . .
133
133
II
ii!
6.3.3 Free Boundary Problems . . . . . . . . . . . . . . . . . .
6.4 Contour Integration . . . . . . . . . . . . . . . . . . . . . . . . .
6.5 Special Functions . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.5.1 The Gamma Function . . . . . . . . . . . . . . . . . . . .
243
248
259
259
4.2 Green's Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 144
4.3 Differential Operators . . . . . . . . . . . . . . . . . . . . . . . . 151 I 6.5.2 Bessel Functions . . . . . . . . . . . . . . . . . . . . . . . 262
" 6.5.3 Legendre Functions . . . . . . . . . . . . . . . . . . . . . . 268

I
4.3.1 Domain of an Operator . . . . . . . . . . . . . . . . . . 151
4.3.2 Adjoint of an Operator . . . . . . . . . . . . . . . . . . . 152 6.5.4 Sine Functions . . . . . . . . . . . . . . . . . . . . . . . . 270
4.3.3 Inhomogeneous Boundary Data . . . . . . . . . . . . . . . 154 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
J Problems for Chapter 6 . . . . . . . . . . . . . . . . . . . . . . . . ~ . 274
4.3.4 The Fredholm Alternative . . . . . . . . . . . . . . . . . . 155 ~
~
4.4 Least Squares Solutions . . . . . . . . . . . . . . . . . . . . . . . 157
4.5 Eigenfunction Expansions . . . . . . . . . . . . . . . . . . . . 161 i
-~
7 Transform and Spectral Theory
7.1 Spectrum of an Operator . . . . . . . . . . . . . . . . . . . . . .
283
283
4.5.1 'Irigonometric Functions . . . . . . . . . . . . . . . . . .
4.5.2 Orthogonal Polynomials . . . . . . . . . . . . . . . . . .
.
.
164
167
I~ 7.2 Fourier Transforms . . . . . . . . , . . . . . . . . . . . . . . . . . 284
~ 7.2.1 Transform Pairs . . . . . . . . . . . . . . . . . . . . , .. 284
4.5.3 Special Functions . . . . . . . . . . . . . . . . . . . . . . . 169
4.5.4 Discretized Operators . . . . . . . . . . . . . . . . . . . . § 7.2.2 Completeness of Hermite and Laguerre Polynomials . . . 297
169
Further R.eading . . . . . . . . . . . . . . . . . · · · · · · · · · · · ·
Problems for Chapter 4 . . . . . . . . . . . . . . . . . . . . . . . . .
·
.
171
171
Is" 7.2.3 Sine Functions . . . . . . . . . . . . . . . . . . . . . . . .
7.2.4 Windowed Fourier Transforms . . . . . . . . . . . . . . .
299
300
7.2.5 Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
I 7.3 Related Integral Transforms . . . . . . . . . . . . . . . . . . . . . 307
5 Calculus of V~iations 177
5.1 The Euler-Lagrange Equations . . . . . . . . . . . . . . . . . . . 177 i•
~
7.3.1 Laplace Transform . . . . . . . . . . . . . . . . . . . . . . 307
5.1.1 Constrained Problems . . . . . . . . . . . . . . . . . . . . 180 7.3.2 Mellin Transform . . . . . . . . . . . . . . . . . . . . . . . 308
5.1.2 Several Unknown Functions . . . . . . . . . . . . . . . . . 181 I 7.3.3 Hankel Transform . . . . . . . . . . . . . . . . . . . . . . 309
5.1.3 Higher Order Derivatives . . . . . . . . . . . . . . . . . .
5.1.4 Variable Endpoints . . . . . . . . . . . . . . . . . . . . . .
184
184
I
li
7.4 Z Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.5 Scattering Theory . . . . . . . . . . . . . . . . . . . . . . . . . .
310
312
~
7.5.1 Scattering Examples . . . . . . . . . . . . . . . . . . . . . 318

I
5.1.5 Several Independent Variables . . . . . . . . . . . . . . . . 185
5.2 Hamilton's Principle . . . . . . . . . . . . . . . . . . . . . . . . . 186 7.5.2 Spectral Representations . . . . . . . . . . . . . . . . . . . 325
5.2.1 The Swinging Pendulum . . . . . . . . . . . . . . . . . . . 188 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
Problems for Chapter 7 . . . . . . . . . . . . . . . . . . . . . . . , . . 328
5.2.2 The Vibrating String . . . . . . . . . . . . . . . . . . . . . 189 I Appendix: Fourier Transform Pairs . . . . . . . . . . . . . . . . . . . . 335
5.2.3 The Vibrating Rod . . . . . . . . . . . . . . . . . . . . . . 189 !
5.2.4 Nonlinear Deformations of a Thin Beam . . . . . . . . . . 193 i
5.2.5 A Vibrating Membrane . . . . . . . . . . . . . . . . . . . 194 Jiii 8 Partial Differential Equations 337
5.3 Approximate Methods . . . . . . . . . . . . . . . . . . . . . . . . 195 ~ 8.1 Poisson's Equation . . . . . . . . . . . . . . . . . . . . . . . . . . 339
5.4 Eigenvalue Problems . . . . . . . . . . . . . . . . . . . . . . . . . 198 8.1.1 Fundamental Solutions . . . . . . . . . . . . . . . . . . . . 339
5.4.1 Optimal Design of Structures . . . . . . . . . . . . . . . . 201 8.1.2 The Method of Images . . . . . . . . . . . . . . . . . . . . 343
Further R.eading . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . 202 8.1.3 Transform Methods . . . . . . . . . . . . . . . . . . . . . 344
Problems for Chapter 5 . . . . . . . . . . . . . . . . . . . . . . . . . . 203 8.1.4 Hilbert Transforms . . . . . . . . . . . . . . . . . . . . . . 355
viii CONTENTS CONTENTS ix

8.1.5 Boundary Integral Equations . . . . . . . . . . . . . . . . 357 Problems for Chapter 11 . . . . . . . . . . . . . . . . . . . . . . . . . . 498

8.1.6 Eigenfunctions . . . . . . . . . . . . . . . . . . . . . . . . 359
8.2 The Wave Equation . . . . . . . . . . . . . . . . . . . . . . . . . 365
8.2.1 Derivations . . . . . . . . . . . . . . . . . . . . . . . . . . 365
8.2.2 Fundamental Solutions . . . . . . . . . . . . . . . . . . . . 368
8.2.3 Vibrations . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
8.2.4 Diffraction Patterns . . . . . . . . . . . . . . . . . . . . . 376
8.3 The a:eat Equation . . . . . . . . . . . . . . . . . . . . . . . . . . 380
8.3.1 Derivations . . . . . . . . . . . . . . . . . . . . . . . . . . 380
8.3.2 Fundamental Solutions . . . . . . . . . . . . . . . . . . . . 383
8.3.3 Transform Methods . . . . . . . . . . . . . . . . . . . . . 385
8.4 Differential-Difference Equations . . . . . . . . . . . . . . . . . . 390
8.4.1 Transform Methods . . . . . . . . . . . . . . . . . . . . . 392
8.4.2 Numerical Methods . . . . . . . . . . . . . . . . . . . . . 395
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400
Problems for Chapter 8 . . . . . . . . . . . . . . . . . . . . . . . . . .
12 Singular Perturbation Theory 505
12.1 Initial Value Problems I . . . . . . . . . . . . . . . . . . . . . . . 505
12.1.1 Van der Pol Equation . . . . . . . . . . . . . . . . . . . . 508
12.1.2 Adiabatic lnvariance . . . . . . . . . . . . . . . . . . . . . 510
12.1.3 Averaging . . . . . . . . . . . . . . . , . . . . . . . . . . . 511
12.1.4 Homogenization Theory . . . . . . . , . . . . . . . . . . . 514
12.2 Initial Value Problems II . . . . . . . . . . . , . . . . . . . . . . . 520
12.2.1 Operational Amplifiers . . . . . . . . . . . . . . . . . . . . 521
12.2.2 Enzyme Kinetics . . . . . . . . . . . . . . . . . . . . . . . 523
12.2.3 Slow Selection in Population Genetics . . . . . . . . . . . 526
.~
12.3 Boundary Value Problems . . . . . . . . . . . . . . . . . . . . . . 528
12.3.1 Matched Asymptotic Expansions . . . . . . . . . . . . . . 528
I
~
12.3.2 Flame Fronts . . . . . . . . . . . . . . . . . . . . . . . . . 539
12.3.3 Relaxation Dynamics . . . . . . . . . . . . . . . . . . . . 542

9 Inverse Scattering Transform

402

411 I 12.3.4 Exponentially Slow Motion . . . . . . . . . . . . . . . . . 548

Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551
9.1 Inverse Scattering . . . . . . . . . . . . . . . . . . . . . . . . . .
9.2 Isospectral Flows . . . . . . . . . . . . . . . . . . . . . . . . . . .
411
417
ii Problems for Chapter 12 . . . . . . . . . . . . . . . . . . . . . . . . . . 552

Bibliography 559
9.3 Korteweg-deVries Equation . . . . . . . . . . . . . . . . . . . . . 421 ~
~
9.4 The Toda Lattice . . . . . . . . . . . . . . . . . . . . . . . . . . . 426 I Selected Hints and Solutions 567
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432 I
~I
Problems for Chapter 9 . . . . . . . . . . . . . . . . . . . . . . . . . . 433 Index 596
10 Asymptotic Expansions 437
10.1 Definitions and Properties . . . . . . . . . . . . . . . . . . . . . 437
10.2 Integration by Parts . . . . . . . . . . . . . . . . . . . . . . . . . 440 I
10.3 Laplace's Method . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.4 Method of Steepest Descents . . . . . . . . . . . . . . . . . . . ;
442
449 I
10.5 Method of Stationary Phase . . . . . . . . . . . . . . . . . . . . .
Further Reading . . . . . . . . ~ . . . . . . . . . . . . . . . . . . . . .
Problems for Chapter 10 . . . . . . . . . . . . . . . . . . . . . . . . . .
456
463
463
'I
11 Regular Perturbation Theory 469 !
11.1 The Implicit Function Theorem . . . . . . . . . . . . . . . . . . 469 I
11.2 Perturbation of Eigenvalues . . . . . . . . . . . . . . . . . . . . . 475
11.3 Nonlinear Eigenvalue Problems . . . . . . . . . . . . . . . . . . . 478 I*
11.3.1 Lyapunov-Schmidt Method . . . . . . . . . . . . . . . . . 482 I
~
11.4 Oscillations and Periodic Solutions . . . . . . . . . . . . . . . . . 482
11.4.1 Advance of the Perihelion of Mercury . . . . . . . . . . . 483
11.4.2 Vander Pol Oscillator . . . . . . . . . . . . . . . . . . . . 485
11.4.3 Knotted Vortex Filaments . . . . . . . . . . . . . . . . . . 488
11.4.4 The Melnikov Function . . . . . . . . . . . . . . . . . . . 493
11.5 Hopf Bifurcations . . . . . . _. . . . . . . . . . . . . . . . . . . . . 494
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496
 Preface to First Edition

Applied mathematics should read like good mystery, with an intriguing begin-
ning, a clever but systematic middle, and a satisfying resolution at the end.
Often, however, the resolution of one mystery opens up a whole new problem,
and the process starts all over. For the applied mathematical scientist, there
is the goal to explain or predict the behavior of some physical situation. One
begins by constructing a mathematical model which captures the essential fea-
tures of the problem without masking its content with overwhelming detail.
Then comes the analysis of the model where every possible tool is tried, and
some new tools developed, in order to understand the behavior of the model as
thoroughly as possible. Finally, one must interpret and compare these results
with real world facts. Sometimes this comparison is quite satisfactory, but most
1
1
often one discovers that important features of the problem are not adequately
accounted for, and the process begins again.
Although every problem has its own distinctive features, through the years

I
it has become apparent that there are a group of tools that are essential to the
analysis of problems in many disciplines. This book is about those tools. But
more than being just a grab bag of tools and techniques, the purpose of this
book is to show that there is a systematic, even esthetic, explanation of how
~
-?£ these classical tools work and fit together as a unit.

II Much of applied mathematical analysis can be summarized by the observa-

tion that we continually attempt to reduce our problems to ones that we already
know how to solve.
This philosophy is illustrated by an anecdote (apocryphal, I hope) of a math-
t ematician and an engineer who, as chance would have it, took a gourmet cooking
class together. As this course started with the basics, the first lesson was on
It how to boil a pot of water. The instructor presented the students with two pots

I•
(1
of water, one on the counter, one on a cold burner, while anot_her burner was
already quite hot, but empty.
In order to test their cooking aptitudes, the instructor first asked the engineer
to demonstrate to the rest of the class how to boil the pot of water that was
already sitting on the stove. He naturally moved the pot carefully from the cold
burner to the hot burner, to the appreciation of his classmates. To be sure the
class understood the process, the pot was moved back to its original spot on the
cold burner at which time the mathematician was asked to demonstrate how to
heat the water in the pot sitting on the counter. He promptly and confidently

xi
xii PREFACE TO FIRST EDITION PREFACE TO FIRST EDITION xiii

exchanged the position of the two pots, placing the pot from the counter onto
the cold burner and the pot from the burner onto the counter. He then stepped
back from the stove, expecting an appreciative response from his mates. Baffled
by his actions, the instructor asked him to explain what he had done, and he
replied naturally, that he had simply reduced his problem to one that everyone
already knew how to solve.
We shall also make it our goal to reduce problems to those which we already
know how to solve.
We can illustrate this underlying idea with simple examples, we know that
to solve the algebraic equation 3x = 2, we multiply both sides of the equation
with the inverse of the "operator" 3, namely 1/3, to obtain x = 2/3. The same
is true if we wish to solve the matrix equation Ax = b where
A=(~!). b=(~)·
Namely, if we know A-t, the inverse of the matrix operator A, we multiply both
sides of the equation by A-t to obtain x : A - 1 b. For this problem,
A-t=! (
8 -1
3 -1 )
3 ' x=~(!)·
H we make it our goal to invert many kinds of linear operators, including
matrix, integral, and differential operators, we will certainly be able to solve
many types of problems. However, there is an approach to calculating the
inverse operator that also gives us geometrical insight. We try to transform the
original problem into a simpler problem which we already know how to solve.
Fbr example, if we rewrite the equation Ax= bas T- 1 AT(T- 1 x) = T- 1 b and
choose
- ( 1 -1 ) -1- 1 ( 1 1 )
T- 1 1 I T - 2 -1 1 I
we find that
1
T- AT = ( ~ -~ ) .
With the change of variables y = T- 1 x, g = T- 1b, the new problem looks
like two of the easy algebraic equations we already know how to solve, namely
4yt = 3/2, -2y2 = 1/2. This process of separating the coupled equations
into uncoupled equations works only if T is carefully chosen, and exactly how
this is done is still a mystery. Suffice it to say, the original problem has been
transformed, by a carefully chosen change of coordinate system, into a problem
we already know how to solve.
This process of changing coordinate systems is very useful in many other
problems. For example, suppose we wish to solve the boundary value problem
u"- 2u = l(x) with u(O) = =
u(1r) 0. As we do not yet know how to write
down an inverse operator, we look for an alternative approach. The single most
important step is deciding how to represent the solution u(x). For example, we
might try to represent u and I as polynomials or infinite power series in x, but
we quickly learn that this guess does not simplify the solution process much.
Instead, the ''natural" choice is to represent u and I as trigonometric series,
00 00
u(x) =L u~,: sinkx, l(x) =L 1~.: sin kx.
k=l k=l
Using this representation (i.e., coordinate system) we find that the original dif-
ferential equation reduces to the infinite number of separated algebraic equations
(k 2 + 2)u~,: = -1~.:· Since these equations are separated (the kth equation de-
pends only on the unknown u~.:), we can solve them just as before, even though
there are an infinite number of equations. We have managed to simplify this
problem by transforming into the correctly chosen coordinate system.
For many of the problems we encounter in the sciences, there is a natural way
to represent the solution that transforms the problem ip.to a substantially easier
one. All of the well-known special functions, including Legendre polynomials,
Bessel functions, Fourier series, Fourier integrals, etc., have as their common
motivation that they are natural for certain problems, and perfectly ridiculous
in others. It is important to know when to choose one transform over another.
Not all problems can be solved exactly, and it is a mistake to always look for
exact solutions. The second basic technique of applied mathematics is to reduce
hard problems to easier problems by ignoring small terms. For example, to find
the roots of the polynomial x 2 + x + .0001 = 0, we notice that the equation is
=
very close to the equation x 2 + x = 0 which has roots x -1, and x 0, and =
we suspect that the roots of the original polynomial are not too much different
from these. Finding how changes in parameters affect the solution is the goal
of perturbation theory and asymptotic analysis, and in this example we have
a regular perturbation problem, since the solution is a regular (i.e., analytic)
function of the "parameter" 0.0001.
Not all reductions lead to such obvious conclusions. For example, the poly-
nomial 0.0001x 2 + x + 1 = 0 is close to the polynomial x + 1 = 0, but the first
has two roots while the second has only one. Where did the second root go?
We know, of course, that there is a very large root that "goes to infinity" as
"0.0001 goes to zero," and this example shows that our naive idea of setting all
small parameters to zero must be done with care. As we see, not all problems
with small parameters are regular, but some have a singular behavior. We need
to know how to distinguish between regular and singular approximations, and
what to do in each case.
This book is written for beginning graduate students in applied mathemat-
ics, science, and engineering, and is appropriate as a one-year course in applied
mathematical techniques (although I have never been able to cover all of this
material in one year). We assume that the students have studied at an intro-
ductory undergraduate level material on linear algebra, ordinary and partial
differential equations, and complex variables. The emphasis of the book is a
working, systematic understanding of classical techniques in a modern context.
Along the way, students are exposed to models from a variety of disciplines.
xiv PREFACE TO FIRST EDITION
It is hoped that this course will prepare students for further study of modem
techniques and in-depth modeling in their own specific discipline.
One book cannot do justice to all of applied mathematics, and in an effort
to keep the amount of material at a manageable size, many important topics
were not included. In fact, each of the twelve chapters could easily be expanded
into an entire book. The topics included here have been selected, not only for
their scientific importance, but also because they allow a logical flow to the
development of the ideas and techniques of transform theory and asymptotic
analysis. The theme of transform theory is introduced for matrices in Chapter
one, and revisited for integral equations in Chapter three, for ordinary differ-
ential equations in Chapters four and seven, for partial differential equations in
Chapter eight, and for certain nonlinear evolution equations in Chapter nine.
Once we know how to solve a wide variety of problema via transform theory,
it becomes appropriate to see what harder problems we can reduce to those we-
know how to solve. Thus, in Chapters ten, eleven, and twelve we give a survey
of the three basic areas of asymptotic analysis, namely asymptotic analysis of
integrals, regular perturbation theory and singular perturbation theory.
Here is a summary of the text, chapter by chapter. In Chapter one, we
review the basics of spectral theory for matrices with the goal of understanding
not just the mechanics of how to solve matrix equations, but more importantly,
the geometry of the solution process, and the crucial role played by eigenvalues
and eigenvectors in finding useful changes of coordinate systems. This usefulness
extends to pseudo-inverse operators as well as operators in HUbert space, and
so is a particularly important piece of background information.
In Chapter two, we extend many of the notions of finite dimensional vector
spaces to function spaces. The main goal is to show how to represent objects
in a function space. Thus, Hilbert spaces and representation of functions in a
Hilbert space are studied. In this context we meet classical sets of functions
such as Fourier series and Legendre polynomials, as well as less well-known sets
such as the Walsh functions, Sine functions, and finite element bases.
In Chapter three, we explore the strong analogy between integral equations
and matrix equations, and examine again the consequences of spectral theory.
This chapter is more abstract than others as it is an introduction to functional
analysis and compact operator theory, given under the guise of Fredholm integral
equations. The added generality is important as a framework for things to come.
In Chapter four, we develop the tools necessary to use spectral decompo-
sitions to solve differential equations. In particular, distributions and Green's
functions are used as the means by which the theory of compact operators can
be applied to differential operators. With these tools in place, the completeness
of eigenfunctions of Sturm-Liouville operators follows directly.
Chapter five is devoted to showing how many classical differential equations
can be derived from a variational principle, and how the eigenvalues of a differ-
ential operator vary as the operator is changed.
Chapter six is a pivotal chapter, since all chapters following it require a sub-
stantial understanding of analytic function theory, and the chapters preceding it
require no such knowledge. In particUlar, knowing how to integrate and differ-
PREFACE TO FIRST EDITION XV
entiate analytic functions at a reasonably sophisticated level is indispensable to
the remaining text. Section 6.3 (Applications to Fluid Flow) is included because
it is a classically important and very lovely subject, but it plays no role in the
remaining development of the book, and could be skipped if time constraints
demand.
Chapter seven continues the development of transform theory and we show
that eigenvalues and eigenfunctions are not always sufficient to build a trans-
form, and that operators having continuous spectrum require a generaliZed con-
struction. It is in this context that Fourier, Mellin, Hankel, and Z transforms,
as well as scattering theory for the SchrOdinger operator are studied.
In Chapter eight we show how to solve linear partial differential and differ-
ence equations, with special emphasis on (as you guessed) transform theory. In
this chapter we are able to make specific use of all the techniques introduced so
far and solve some problems with interesting applications.
Although much of transform theory is rather old, it is by no means dead.
In Chapter nine we show how transform theory has recently been used to solve
certain nonlinear evolution equations. We illustrate the inverse scattering trans-
form on the Korteweg-deVries equation and the Toda lattice.
In Chapter ten we show how asymptotic methods can be used to approx-
imate the horrendous integral expressions that so often result from transform
techniques.
In Chapter eleven we show how perturbation theory and especially the study
of nonlinear eigenvalue problems uses knowledge of the spectrum of a linear
operator in a fundamental way. The nonlinear problems in this chapter all have
the feature that their solutions are close to the solutions of a nearby linear
problem.
Singular perturbation problema fail to have this property, but have solutions
that differ markedly from the naive simplified problem. In Chapter twelve, we
give a survey of the three basic singular perturbation problema (slowly varying
oscillations, initial value problems with vastly different time scales, and bound-
ary value problems with boundary layers).
This book has the lofty goal of reaching students of mathematics, science,
and engineering. To keep the attention of mathematicians, one must be system-
atic, and include theorems and proofs, but not too many explicit calculations.
To interest an engineer or scientist, one must give specific examples of how to
solve meaningful problema but not too many proofs or too much abstraction.
In other words, there is always someone who is unhappy.
In an effort to minimize the total displeasure and not scare off too many
members of either camp, there are some proofs and some computations. Early
in the text, there are disproportionately more proofs than computations because
as the foundations are being laid, the proofs often give important insights. Later,
after the bases are established, they are invoked in the problem solving process,
but proofs are deemphasized. (For example, there are no proofs in Chapters
eleven and twelve.) Experience has shown that this approach is, if not optimal,
at least palatable to both sides.
This is intended to be an applied mathematics book and yet there is very
xvi PREFACE TO FIRST EDITION
little mention of numerical methods. How can this be? The answer is simple
and, I hope, satisfactory. This book is primarily about the principles that one
uses to solve problems and since these principles often have consequence in
numerical algorithms, mention of numerical routines is made when appropriate.
However, FORTRAN codes or other specific implementations can be found in
many other good resources and so (except for a code for fast Walsh transforms)
are omitted. On the other hand, many of the calculations in this text should
be done routinely using symbolic manipulation languages such as REDUCE or
MACSYMA. Since these languages are not generally familiar to many, included
here are a number of short programs written in REDUCE in order to encourage
readers to learn how to use these remarkable tools. (Some of the problems at
the end of several chapters are intended to be so tedious that they force the
reader to learn one of these languages.)
This text would not have been possible were it not for the hard work and
encouragement of many other people. There were numerous students who strug-
gled through this material without the benefit of written notes while the course
was evolving. More recently, Prof. Calvin Wilcox, Prof. Frank Hoppensteadt,
Gary deYoung, Fred Phelps, and Paul Arner have been very influential in the
shaping of this presentation. Finally, the patience of Annetta Cochran and
Shannon Ferguson while typing from my illegible scrawling was exemplary.
In spite of all the best efforts of everyone involved, I am sure there are still
typographical errors in the text. It is disturbing that I can read and reread
a section of text and still not catch all the erors. My hope is that those that
remain are both few and obvius and will not lead to undue confusion.
Preface to the Second
Edition
When the first edition of this book was published, l thought that it was an
up-to-date look at classical methods of applied mathematics. I did not expect
the basic tools of applied mathematicians to change dramatically in the near
future. Indeed the basic ideas remain the same, but there are many ways in
which those basic ideas have found new applications and extensions.
Some of the most significant new techniques of the last decade are wavelet
analysis, multigrid methods, and homogenization theory, all of which are exten-
sions of methods already discussed in the first edition. Additionally, software
tools have become much more sophisticated and reliable, and it is not possible
to ignore these developments in the training of an applied mathematician.
So the first reason to revise this book is to bring it up to date, by adding
material describing these new and important methods of applied mathematics.
The second reason for this revision is to make it easier to read and use. To
that end, the text has been thoroughly edited, with emphasis on clarity and
filling in the gaps. I have tried to eliminate those annoying places where a step
was described as obvious, but is far from obvious to the reader. I have also tried
to eliminate many of the (mostly typographical, but nonetheless noisome) errors
that were in the first edition. I have added equation numbers to those equations
that are referenced in the text, and many figures have been added to aid the
geometrical interpretation and understanding. Finally, I have added a section
of hints and solutions for the exercises. Many of the exercises are difficult and
both students and instructors find such a section to be most helpful.
Many new exercises have been added, mostly those which are intended to use
modern software for symbolic computation and graphical interpretation. The
first edition of this book referred to REDUCE, which, sadly, disappeared from
widespread use almost immediately after the publication of that edition. The
good news is that the new software tools are much better. My personal pref-
erences are Maple and Matlab, but other choices include Mathematica, Math-
Cad, Derive, etc. Exercises that are marked with .II!, are designed to encourage
computer usage with one of these software packages. I have mentioned Maple
throughout the text, but the intent is "generic software package with symbolic
computation and graphics capability".
xvii
 PREFACE TO SECOND EDITION xix
xviii PREFACE TO SECOND EDITION

While the basic philosophy of the text remains unchanged, there are impor-
tant ways in which most chapters have been modified. In particular, here are
some of the major modifications and additions by chapter:
Chapter 1. A section describing the problem of Procrustes (who devised an
unusual method for coordinate transformations) has been added.
Also, a section on applications of eigenvalues and eigenfunctions
of matrices has been added, wherein matrix iterative methods and
the ranking of sports teams are discussed.
folks at Calvin College for providing me with a quiet place to work for several
months, and to the University of Utah for a sabbatical leave during which this
revision was finished.
Further corrections or modifications can be found at the Internet site
http://www.math.utah.edu/-keener/AppliedMathBook.html.
James P. Keener
University of Utah
June 1999

Chapter 2. A section introducing wavelets has been added, replacing the dis-
cussion of Walsh functions.
Chapter 4. The discussion of extended operators, which I always found to be
awkWard, is replaced by a much more transparent and straight-
forward discussion of inhomogeneous boundary conditions using
integration by parts.

Chapter 5. An example of how variational principles are used to design optimal

structures has been added.

Chapter 7. Further development and construction of wavelets, as well as other

transforms (such as windowed Fourier transforms) is included.
Chapter 8. A discussion of the Hilbert transform and boundary integral meth-
ods has been added, along with some interesting new applications.
There is also added emphasis on diffusion-reaction equations, with
a discussion of the Turing instability, and derivations of the Calm-
Allen and Calm-Hilliard equations. Finally, a discussion of multi-
grid methods for the numerical solution of Poisson's equation has
been added.
Chapter 11. A discussion of the Lyapunov-Schmidt technique for bifurcation
problems has been added, as has a description of the Melnikov
function.
Chapter 12. The discussion of the averaging theorem has been expanded, while
discussions of homogenization theory and exponentially slow mo-
tion of transition layers have been added.

The content of Chapters 3, 6, 9 and 10 has changed little from the first
edition.
As with any project of this size, I owe a debt of gratitude to many people who
helped. Eric Cytrynbaum, Young Seon Lee, Todd Shaw and Peter Bates found
many of the errors that I missed and made numerous helpful suggestions about
the presentation, while David Eyre and Andrej Cherkaev made many helpful
suggestions for new material to include. Steve Worcester did the initial 'J.EX-ing
of the manuscript in record time. Nelson Beebe is a 'J.EX-pert, whose wizardry
never ceases to astound me and who8e help was indispensable. Thanks to the
Chapter 1

Finite Dimensional Vector

Spaces

1.1 Linear Vector Spaces

In any problem that we wish to solve, the goal is to find a particular object,
chosen from among a large collection of contenders, which satisfies the governing
constraints of the problem. The collection of contending objects is often a
vector space, and although individual elements of the set can be ca.lled by many
different names, it is common to ca.ll them vectors. Not every set of objects
constitutes a vector space. To be specific, if we have a collection of objects S,
we must first define addition and sca.la.r multiplication for these objects. The
operations of addition and sca.lar multiplication are defined to satisfy the usual
properties: H x,y,z E S, then
• x +y = y + x (commutative law)
• x + (y + z) = (x + y) + z (associative law)
• 0 E S, 0 + x = x (additive identity)
• -xES, -x + x = 0 (additive inverse)
and if xES, a, {3 E IR (or <U) then
• a({jx) = (a{j)x
• (a + {j)x = ax + {jx
• a(x + y) = ax+ ay
• lx = x, Ox =0.

Once the operations of addition and scalar multiplication are defined, we

define a vector space as follows:

1
2 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES
Definition 1.1 A set of objects Sis called a linear vector space if two prop-
erties hold:
1. If x, y E S, then x +y E S (called closure under vector addition),
2. If a E IR (or a E ([;') and x E S, then ax E S (called closure under scalar
multiplication).
If the scalars are real, S is a real vector space, while if the scalars are complex,
S is a complex vector space.
Examples:
1. The set of real ordered pairs (:z:,y) (denoted m?) is a linear vector space
if addition is defined by (:z:1, yt) + (:z:2, y2) = (:z:1 + x2, Yt + y2), and scalar
multiplication is defined by a(:z:,y) = (a:z:,ay).
2. The set of real n-tuples (:z:t, x2, ... , :Z:n) (denoted lRn) forms a linear vector
space if addition and scalar multiplication are defined component-wise as
in the above example. Similarly the set of complex n-tuples (denoted <IJR)
forms a linear vector space with the complex scalars.
3. The set of all polynomials of degree n forms a linear vector space, with
"vectors" Pn(:z:) = :E.i=o ai:z:i, with Clj E 1R (or Clj E IV), if addition and
scalar multiplication are defined in the usual way.
4. The set of all continuous functions defined on some interval [a, b] forms a
linear vector space.
There are many sets that are not linear vector spaces. For example, the
set of all ordered pairs of the form (x, 1) is not closed under addition or scalar
multiplication and so does not form a vector space. On the other hand, ordered
pairs of the form (x, 3x) do comprise a linear vector space. Similarly, the set of
continuous functions /(x) defined on the interval [a,b) with f(a) =
1 does not
form a vector space, while if instead the constraint /(a) = 0 is imposed, the
properties of closure hold.
In a vector space, we can express one element of S in terms of additions
and scalar multiplications of other elements. For example, if Xt. x2, ... , Xn, are
elements of S, then x = a1x 1 + a2x 2 + · · · + anXn is also inS, and xis said to
be a linear combination of Xt, x2, ... , Xn· If there is some linear combination
=
with a1x 1 + a 2x 2 + · · · + anXn 0, and not all of the scalars ai are zero, then
the set {x 1 , x 2 , ••• , Xn} is said to be linearly dependent. On the other hand,
if the only linear combination of Xt, x2, ... , Xn which is zero has all ai equal to
zero, the set {x 1 , x 2 , •.• , Xn} is said to be linearly independent.
Examples:
1. The single nontrivial vector x E lRn, :z: ::j: 0 forms a linearly independent
set because the only way to have a:z: = 0 is if a = 0.
1.1. LINEAR VECTOR SPACES 3
2. A linear combination of two vectors in m?, Clt(Xt,Yl) + Cl2(:Z:2,Y2), is the
zero vector whenever :z:1 = -/3:z:2 and Yt = -f3y2 where f3 = a2/a1. Geo-
metrically, this means that the vector (:z:t, yl) is collinear with the vector
(:z::~, y2), i.e., they are parallel vectors in 1R2.
Important examples of linearly independent vectors are the monomials. That
is, the powers of x, {1, x, x 2 , ••• , xn} form a linearly independent set. An easy
proof of linear independence is to define f(x) = ao + alx + a2x 2 + ... + anxn
and to determine when f(x) is identically zero for all x. Clearly, /(0) = ao and
if f(x) = = =
0 for all x, then /(0) 0 implies a0 0. The kth derivative of f(x)
at x = 0,
dk f(x)
d,xk
I
z=O
= k!ak
is zero if and only if ak = 0. Therefore, f(x) =
0 if and only if ai = 0 for
j = 0, 1, 2, ... , n.
It is often convenient to represent an element of the vector space as a linear
combination of a predetermined set of elements of S. For example, we might
want to represent the continuous functions as a linear combination of certain
polynomials or of certain trigonometric functions. We ask the question, given
a subset of S, when can any element of S be written as a linear combination of
elements of the specified subset?
Definition 1.2 A set of vectors T C S is a spanning set if every x E S can
be written as a linear combination of the elements ofT.
Definition 1.3 The span of a set of vectors T is the collection of all vectors
that are linear combinations of the vectors in T.
Definition 1.4 If a spanning set T c S is linearly independent then T is a
basis for S. If the number of elements ofT is finite, then S is a finite dimensional
vector space whose dimension is the number of elements ofT.
One can easily show (see Problem 1.1.1) that every basis of S has the same num-
ber of elements. For a given basis T, a vector inS has a unique representation
in terms of the elements of T.
Examples:
1. In 1R2 , any three vectors are linearly dependent, but any two noncollinear
vectors form a basis.
2. In 1R 3 the vectors { (1, 0, 0), (0, 1, 0), (0, 0, 1), (1, 1, 0), (0, 1, 1)} form a span-
ning set, but the last two vectors are not necessary to form a spanning set.
From this set, there are eight different ways to choose a subset of 3 linearly
independent elements which is a basis. In lRn, the so-called natural basis
is the set of vectors {et, e2, ... , en} where the jth entry of e~c is Okj where
o,i = 0 if j ::j: k and O~ok = 1. The object o"i is called the Kronecker
delta.
 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.1. LINEAR VECTOR SPACES 5
4

3. The set {1,z,z2 ,z2 -1} is linearly dependent. However, any of the subsets similarly we discretize g(x) to define G = (g(zl),g(z2), ... ,g(xn)), where
{1,z,z2}, {1,z,z2 - 1}, or {z,z2,z 2 - 1} forms a basis for the set of :z:,. = kfn,
and it makes sense to define
quadratic polynomials. 1 n

4.· The set of all polynomials does not have finite dimension, since any linear (F, G)n = - 2: f(x~c)g(xlc)·
n k=l
combination of polynomials of degree n or less cannot be a polynomial of
degree n + 1. The set of all continuous functions is also infinite since the (While this is an inner product on JR", this is not ap inner product for the
polynomials are a subset of this set. We will show later that the polynomials continuous functions. Why?) Taking the limit n -+ oo, we obtain
form a "basis" for the continuous functions, but this requires more technical
information than we have available now. (/,g} = 1 1
f(x)g(x)dz. (1.2)

When you first learned about vectors in an undergraduate Physics or Math- It is an easy matter to verify that (1.2) defines an inner product on the
ematics course, you were probably told that vectors have direction and magni- vector space of continuous functions.
tude. This is certainly true in JR3 , but in more complicated vector spaces the
3. There are other interesting ways to define an inner product for functions.
concept of direction is hard to visualize. The direction of a vector is always given For example, if the complex valued functions I and g are continuously
relative to some other reference vector by the angle between the two vectors. differentiable on the interval [0, 1], we might define
To get a concept of direction and angles in general vector spaces we introduce
the notion of an inner product. (/,g)= 1 1
(!(z)g(x) + /'(z)gl(z)) dz.
Definition 1.5 For x, y E S, the inner product (also called scalar product More generally, if I and g are n times continuously differentiable on [0, 1]
or dot product) of x andy, denoted (x,lf), is a function{-,·): S x S ~ IR, we might define
(or (U if Sis a complex vector space) with the properties:

1. (x,y) = (y,z), (overline denotes complex conjugate)

(f,g} = r (t
lo \j,.,0
di !(~)dig(~)) dx
dx1 dxJ
(1.3)

2. (ax,y) = a(x,y}, as an inner product. Indeed, one can show that these satisfy the required
properties of inner products.
3. (x+y,z)=(x,z)+(y,z),
The magnitude of a vector can also be defined for a general vector space.
4. (x,x) > 0 if x :/: 0, (x,x) = 0 if and only if x = 0.
Definition 1.6 For x E S, the norm (also called amplitude, magnitude, or
A linear vector space with an inner product is called an inner product length) of x, denoted llxll, is a function 11·11: S-+ [O,oo) with the properties:
space.
1. llxll > 0 if x :/: 0 and llxll = 0 implies x = 0,
Examples: 2. llaxll = lal·llxll for a E IR (or a E <U if Sis a complex vector space),
1. In JR.", suppose z = (zt, :1:2, ... , Zn) and Sl =(Sit, f12, ... , Sin) then 3. llx + Yll $ llxll + IIYII (triangle inequality).
n

<x, 11> = E x,y, In JRn, we can take llxll = (L:~= 1 Ixkl 2 ) 1 1 2 or more generally, llxll =
lo•l (L:~= 1 1xkiP) 1 1P, 1 < p < oo. With p = 2, this is the Euclidean norm or
"2"-norm, however, for differing values of p the meaning of size changes. For
is the usual Euclidean inner product. H x, Sl e (U" I then = =
example, in JR2 the "unit sphere" llxll 1 with p 2 is the circle x~ + x~ 1 =
n while with p = 1 the "unit sphere" lx1l + lx2l = 1 is a diamond (a square with
(:~:, y) = E :l:loYio (1.1) vertices on the axes). H we let p ~ oo, then llxll = maxk lxkl llxlloo is called =
lo=l the "infinity" or "sup" (for supremum) norm, and the "unit sphere" llxll = 1 is
is an inner product. a square (Fig. 1.1).
A linear vector space with a norm is called a normed vector space. Al-
2. For two real continuous functions l(x) and g(z) defined for :1: e [0,1], we
though there are many ways to define norms, one direct way to find a norm is
discretize l(x) to define a vector in mn
IF= (f(xt), l(z2), ... ,/(xn)) and
 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.1. LINEAR VECTOR SPACES 7
6

If y =f
0, we pick a= fMtt,
(thereby minimizing llx- ay!l 2) from which the
Schwarz inequality is immediate. 1
Using the Schwarz inequality, we see that the triangle inequality holds for
any induced norm, since

llx + Yll 2 = llxW + 2Re{ (x, y)) + IIYII 2

(a) (b) (c)
2
~ llxll 2 + 2llxll·llvll + llvll
Figure 1.1: The unit sphere llxll = 1 for (a) the "1"-norm, {b) the "2"-norm,
(c) the sup norm. = (llxll + llviD 2,
or
if we have an inner product space. If x E Sand (x,y} is an inner product for llx +vii ~ llxll + IIYII·
=
S then llxll J(x,x} is called the induced norm (or natural norm) for S. 2
One can show that in JR , cos(}= ~ where (x, y) is the usual Euclidean
For example, in (Jln, the Euclidean norm
inner product and(} is the angle between the vectors x andy. In other real inner

t; lx~~:l )1/2
n product spaces, such as the space of continuous functions, we take this to be
2 {1.4)
llxll = (
, the definition of cosO. The law of cosines and the Pythagorean theorem are
immediate. For example,
is induced by the Euclidean inner product {1.1), and for real continuous func- llx + vW = llxll 2 + 2llxll . IIYII cos(} + IIYW
tions with inner product (/,g)= J:
f(x)g(x)dx, the induced norm is
is the law of cosines, and if cos() = 0 then

11/11 =
b
L1f(x)l dx
)1/2
2 llx + Yll 2 = 2
llxll + IIYW (Pythagorean Theorem).
(
Following this definition of cos (), we say that the vectors x and y are or-
For n times continuously differentiable complex valued functions, the inner prod- thogonal if (x, y) = 0.

r
uct (1.3) induces the norm
Examples:

1\/1\ = (!.' ~ ldl£,<;l r dz product since ((0, 1), (1, 0)} = 0.

JR? using the Euclidean inner
1. The vector (1, 0) ia orthogonal to (0, 1) in

For any induced norm, llxll > 0 if x :/: 0, and llaxll = lal· llxll for scalars a.
To see that the triangle inequality also holds, we must first prove an important
result:
Theorem 1.1 (Schwarz 1 inequality) For x,y in an inner product space,
l(x, v>l ~ llxii·IIYII·
2. With inner product (!,g)= J;.,. f(x)g(x)dx, sinx ap.d sin2x are orthogonal
·
smce r"J1r smxsm
Jo · · 2x dx = 0.
3. With inner product (!,g)= J01 f(x)g(x)dx, the angle 0 between the func-
tions f(x) = 1 and g(x) = x on [0, 1) is 30° since cosO= ~ = .;'3/2.
Orthogonal vectors are nice for several reasons. If the vectors {¢1, ¢2, ... ,
{1.5) ¢n} E S, cfoi =f 0, are mutually orthogonal, that is, (¢i,¢;) = 0 fori =f j,
then they form a linearly independent set. To see this suppose there are scalars
a1, a2, ... , an so that

a1cfo1 + a2¢2 + · · · + ancfon = 0.

Proof: For x, yES, a a scalar,
The inner product of this expression with ¢; is
0~ llx- ayll 2 = llxll 2 - 2
2Re((x, ay}) + lai IIYII
2
•
1 This theorem is also associated with the names of Cauchy and Bunyakowsky. (¢;, 01¢1 + 02¢2 + · · · + ancfon) = (¢;,a;¢;) = 0:; llc/Jj 11 2 = 0,
8 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.2. SPECTRAL THEORY FOR MATRICES 9

=
so that ai 0 for j = 1, 2, ... , n, provided t/J; is nontrivial.
H the set {1/Jt. t/J2, ... , t/Jn} forms a basis for S, we can represent any element
f E S as a linear combination of 1/J;, .
~\),
n \

I= Lf3itP3· \, q, 2
\
j=l

To determine the coefficients /3;, we take the inner product of f with tPi and
find a matrix equation B/3 =
'7 where B = (bi;), btj = (1/Jj,tPi}, {3 = ({3i), Xz
'1i = (/, t/Jt}. This matrix problem is always uniquely solvable since the tPt 's
are linearly independent. However, the solution is simplified enormously when Figure 1.2: Graphical representation of the Gram-Schmidt orthogonalization
the tPi 's are mutually orthogonal, since then B is a diagonal matrix and easily procedure to produce tP2 from Xt = c/>1 and x2.
inverted to yield
(!, tPi}
{3 Example:
i = llt/Jill
2 •
Consider the powers of x, {1, x, x 2 , ••• , xn}, which on any interval [a, b] (a<
The coefficient f3t carries with it an appealing geometrical interpretation in b) form a linearly independent set.
m,n. H we want to qrthogonally "project" a vector f onto ¢ we might imag- With [a,b] = [-1, 1] and the inner product
ine shining a light onto tjJ and measuring the shadow cast by f. From simple
1
trigonometry, the length of this shadow is II/II cosO where 8 is the angle in the (!,g) =[ f(x)g(x)dx,
plane defined by the vectors f and 1/J. This length is exactly ~ and the vector 1

of the shadow, ~1/J, is called the projection off onto 1/J. Notice that the the Gram-Schmidt procedure produces
quantity llx- 'i'YII is minimized by picking 'i'Y to be the projection of x onto y, if>o = 1,
. ~
1.e., 'i' = TTUfP"Y· l/>1 =x,
The Gram-Schmidt orthogonalization procedure is an inductive tech- l/>2 = x 2 -1/3,
l/>3 = x3 - 3x/5,
nique used to generate a mutually orthogonal set from any linearly independent
=
set of vectors. Given the vectors Xt, x2, ... , Xn, we set t/J1 Xt. To find a vector and so on. The functions thus generated are called the Legendre poly-
t/J2 that is orthogonal to t/J1, we set t/J2 = x2 - a¢1, and use the requirement nomials. Other sets of orthogonal polynomials are found by changing the
=
(t/J2, t/J1) 0 to determine that underlying interval or the definition of the inner product. We will see more
of these in Chapter 2.

1.2 Spectral Theory for Matrices

so that
{x2,tPt) In a vector space, there are many different possible choices for a basis. The main
tP2 = X2- llt/Jtll2 tPt• message of this section and, in fact, of this text, is that by a careful choice of
basis, many problems can be "diagonalized." This is also the main idea behind
In other words, we get tjJ 2 by subtracting from x2 the projection of x2 onto tPt transform theory.
(See Fig. 1.2}. Proceeding inductively, we find t/Jn by subtracting the successive Suppose we want to solve the matrix problem
projections,
n-1 ( ) Ax=b,
.J. ~ Xn, tPi t/J;
'l'n = Xn - LJ 111/J 112 '
j=1 j where A is an n x n matrix. We view the entries of the vectors x and b as the
coordinates of vectors relative to some basis, usually the natural basis. That is,
from which it is clear that (1/J~;, t/Jn) =0 for k < n. We are left with mutually
n
orthogonal vectors c/>1, t/J2, .•. , c/>n. which have the same span as the original set.
x = (x1,x2, ... ,xn)T = L.x;ef.
j=l
10 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES
The same vector x would have different coordinates if it were expressed relative
to a different basis.
Suppose the matrix equation Ax = b is expressed in coordinates relative
to the basis {t/J1 , '1/J?., ••• , t/Jn} and we wish tore-express the problem relative to
some other basis {tPb tP2, ... , t/>n}. What does this change of basis do to the
original representation of the matrix problem?
Since {t/Ji} forms a basis, for each vector t/>;, there are numbers c~i), i =
1, 2, ... , n for which t/>; = c~i) tPi. The numbers c~i) are the coordinates of
E;=t
the vector t/>; relative to the basis {t/JJ}· In the case that {tPJ} is the natural
basis, the number c~i) is the jth element of the vector tj>;.
H $; and :z:~ are coordinates of a vector x relative to the bases {t/J;} and {tPi},
respectively, then
x = I;x~:t/J~c
n
= 'Ex~tj>;
n
= I;x~ I;c~')t/Ji = 2:; 'Ec~'>x~ t/JJ,
n (n ) n (n )
A:=l i=t i=t j=l i=l i=l
so that, since the representation of a vector in terms of a given basis is unique,
ZJ = E~ 1 c~')x~. Written in vector notation,$= C:z:' where Cis the matrix
of coefficients that expresses the basis {t/>;} in terms of the basis {t/J;}, with the
number c~') in the jth row and ith column of C.
Now the original problem A$= b becomes A'x' = c-tACx' = b', where
A' = c-tAC, z, bare the coordinates of x,b relative to the original basis
{t/J;}, and x', b' are the coordinates of :z:, b relative to the new basis {1{1;}. The
transformation A' = c- 1 AC is called a similarity transformation, and we
see that all similarity transformations. are equivalent to a change of basis and
vice versa. Two matrices are said to be equivalent or similar if there is a
similarity transformation between them.
This transformation can be made a bit clearer by examining what happens
when the original basis is the natural basis. H AE is the representation of A
with respect to the natural basis and {t/>;} is the new basis, then 0 1 is the matrix
whose columns are the vectors tj>;. The representation of A with respect to {t/>;}
is At = 01t AECt. Similarly, if C~ has as its columns the basis vectors {t/Ji},
then the representation of A with respect to {t/J;} is A~ = C:J 1AEC~. Thus, if
we are given A~, the representation of A with respect to {t/J;}, and wish to find
At, the representation of A with respect to {t/>1}, we find that
At= 01t AECt = C1 1 (C~A~c;t)Ct = (C;tCt)- 1 A~(c;tCt)·
In other words, the matrix C that transforms A~ to At is C = c;tct where
Ct has t/>; in its ith column and C~ has t/J; in its ith column.
Under what conditions is there a change of basis (equivalently, a similarity
transformation) that renders the matrix A diagonal? Apparently we must find
a matrix C so that AC = C A where A is diagonal, that is, the column vectors
of C, say :z:;, must satisfy A:z:, = ~;:z:;, where A; is the ith diagonal element of
A.
1.2. SPECTRAL THEORY FOR MATRICES 11
Definition 1. 7 An eigenpair of A is a pair (A, :z:), A E <V, :z: E <lin satisfying
Ax = A:z:, :z: i- 0.
The vector :z: is called the eigenvector and A is called the eigenvalue of A.
A number.>. is an eigenvalue of A if and only if the equation (A- >.I)x = 0
has a nontrivial solution, that is, if and only if the matrix A- >.I is singular. It
follows that A is an eigenvalue of A if and only if it is a root of the nth order
polynomial
PA(A) = det(A- >.I), (1.6)
called the characteristic polynomial for A.
There are always n roots, counting multiplicity, of an nth order polynomial.
The order of any particular root is called its algebraic multiplicity while the
number of linearly independent eigenvectors for a given eigenvalue is called its
geometric multiplicity. The geometric multiplicity can be no greater than
the algebraic multiplicity for an eigenvalue A.
Our motivation for finding eigenvectors is that they provide a way to repre-
sent a matrix operator as a diagonal operator. Another important interpretation
of eigenvectors is geometrical. If we view the matrix A as a transformation that
transforms one vector into another, then the equation Ax = .>.:z: expresses the
fact that some vector, when it is transformed by A, has its direction unchanged,
even though the length of the vector may have changed. For example, a rigid
body rotation (in three dimensions) always has an axis about which the rotation
takes place, which therefore is the invariant direction.
Theorem 1.2 Suppose A is an n x n matri:z:.
1. If the matri:z: A has n linearly independent real (or complex} eigenvectors,
there is a real (or complex) change of basis in Jl(l (or <lin) so that relative
to the new basis A is diagonal.
2. lfT is the matri:z: whose columns are the eigenvectors of A, then T- 1 AT=
A is the diagonal matri:z: of eigenvalues.
The factorization A= TAT- 1 is called the spectral representation of A.
Proof: Suppose :z:1, X2, ... , :Z:n are linearly independent eigenvectors of A, with
Axi = Ai:z:;. LetT be the matrix with columns of vectors Xi. Then we have
Xn)[ ~ {A:z:1 A:z:2 ... Ax]n) = (A1:z:1 A2:z:2 ... Ana:n)
AT = A(:z:1 :z:2 ...
1
.>.2 0
= (:z:1 :z:2 ... :z:n) = TA,
0 An
where A is diagonal, or
T- 1 AT=A.
I
12 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.2. SPECTRAL THEORY FOR MATRICES 13

Examples: Proof: The proof of this statement is by induction. For each eigenvalue >..~:,
we let x.~: be the corresponding eigenvector. For k = 1, the single vector
1. The matrix x1 =F 0 forms a linearly independent set. For the induction step, we suppose
A=(~!) x1,x:l,· .. ,x.~:-1 are linearly independent. We try to find scalars a1, a2, ... ,a.~:
has eigenpairs ~~ = 4, =
XI (1,1)T, and ~2 x2=2, =
(1,-1)T 2 • The
vectors X1, x2 are real and linearly independent. The matrix
so that
a1x1 + a2x2 + · · · + a~cx1c = 0.
If we multiply this expression by A and use that Axi = AiXi, we learn that
T= ( 1 1)
1 -1 a1>.1x1 + a2>.2x2 + · · · + a~c>.~cx~c = 0.
gives the required change of basis and
If we multiply instead by >..~:, we obtain
T- 1 AT= ( 0 4 0) 2 . >.~ca1x1 + >.1ca2x2 + · · · + >.~ca~cx~c = 0.
2. The matrix Subtracting the second of these expressions from the first, we find that
A= ( c?s(} -sin(} )
smO cos(} a1(>.1- >..~:)xl + · · · + a.~:-1(>..~:-1- >..~:)x.~:-1 = 0.
transforms the vector (cos¢, sin c/J)T into the vector (cos(¢+0),sin(¢+0))T
and so is called a rotational matrix. This matrix has eigenvalues ~ = It follows that a1 = = ··· =
a2 a1c-1 =
0, since xl,X2,···•XA:-l are assumed
sin(}± icos(} and eigenvectors (=Fi, l)T. There is no real change of basis to be linearly independent and Ale is distinct from >.1, >.2, ... , Aie-l, and finally
which diagonalizes A, although it can be diagonalized using complex basis a.~:= 0 since x~c is assumed to be nonzero.
vectors. I
3. The matrix There is another important class of matrices for which diagonalization is
A=(~~) always possible, namely the self-adjoint matrices, but before we can discuss
these, we must define the adjoint of a matrix.
has characteristic polynomial ~ 2 = 0, so there is one eigenvalue .>. = 0
with algebraic multiplicity two. However, there is only one eigenvector Definition 1.8 For any matrix A, the adjoint of A is defined as the matrix
x 1 = (1, O)T, so the geometric multiplicity is one. There is no change of
A* where (Ax,y) = (x,A*y), for all x, yin (i;ffl.
basis which diagonalizes A.

We need to determine when there are n linearly independent eigenvectors. As Definition 1.9 If A= (a;J), the transpose of A is AT= (a;,).
we see from the last example, the dilemma is that even though the characteristic To find the adjoint matrix A • explicitly, note that (using the Euclidean inner
polynomial (1.6) always has n roots, counting multiplicity, there need not be an product)
equal number of linearly independent eigenvectors. It is this possible deficiency
that is of concern. For any eigenvalue >., the geometric multiplicity is at least (Ax, y)
n,n
= L L aiJXJfii = L Xj
n (nL aiJili
)
one, since det(A- >.I) = 0 implies there is at least one nontrivial vector x for i=l j=l j=l i=l
which (A- >.I)x = 0. In fact, every square matrix A has at least one eigenpair
(although it may be complex).
so that A • = AT. For example, if
Theorem 1.3 If A has n distinct eigenvalues, then it has n linearly independent au a1:l )
eigenvectors. A= a21 a22 ,
(
aa1 aa2
2 Remark about notation: When dealing with vectors and matrices, it is important to then
distinguish between row vectors and column vectors. For example, (1, 2) is a row vector and A• =( au ~~ aa1 ) .
( ~ ) is a column vector. However, it is often convenient to write column vectors as the Cii2 a:l2 aa:l

transpose of a row vector, for example, ( ~ ) = (1, 2)T. Definition 1.10 A matrix A is self-adjoint if A • = A.
14 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.2. SPECTRAL THEORY FOR MATRICES 15

H A is self-adjoint and complex, then A is called a Hermitian matrix
whereas, if A is self-adjoint and real, it is symmetric. Notice that by definition,
a self-adjoint matrix is square.
Theorem 1.4 If A is self-adjoint the following statements are true:
1. (Ax, x) is real for all x,
We observe that if M is an invariant manifold over the complex scalar field
for some matrix A, there is at least one vector x E M with Ax = ..\x for
some ..\ E a::'. To verify this, notice that since M lies in a k-dimensional space,
it has a basis, say {xi. x2, ... , xk}· For any x E M, Ax E M so x and Ax
have representations relative to the basis {xi, x2, ... , Xk}· In particular, take
x == I:~=l aixi and Axi = I:7=l f3jiXj· To solve Ax- ..\x = 0, we must have

f. All eigenvalues are real, k k

0 = L:ai(Axi)- ..\ L:aixi
9. Eigenvectors of distinct eigenvalues are orthogonal, i=l i=l

4. There is an orthonormal basis formed by the eigenvectors,

k (k ) k k
5. The matrix A can be diagonalized. = ~ai j;f3jiXj- ..\xi = t;xi ~(f3ji- AOij)ai.

Observing that {XI, x2, ... , Xk} is a linearly independent set, we have
The proof of this theorem is as follows:
k
1. H A= A*, then (Ax,x) = (x,A*x) = (x,Ax) = (Ax,x), so (Ax,x) is real. L(f3ii- Mii)ai = 0, j = 1,2, ... ,k,
2. H Ax= Ax, then (Ax,x) = (Ax,x} = ,\(x,x). Since (Ax,x} and (x,x} are i=I

real, ,\ must be real.
3. Consider the eigenpairs ( ..\, x) and (#-', y). Then
..\(x, y) = (.\x, y) = (Ax, y) = (x, Ay) = (x, 1-'Y} = J.l(x, y)
so that
= 0.
(,\- J.l){X, y)
H ,\and 1-' are distinct(,\ f: J.l), then (x, y) = 0, i.e., x andy are orthogonal.
The proof of item 4 requires more background on linear manifolds.
Definition 1.11
1. A linear manifold M C S is a subset of S which is closed under vector
addition and scalar multiplication.
2. An invariant manifold M for the matrix A is a linear manifold M C S
for which x E M implies that Ax E M.
Examples:
1. N(A), the null space of A, is the set of all x for which Ax= 0. If x andy
satisfy Ax= 0 and Ay = 0, then A( ax+ {3y) = 0 so that N(A) is a linear
manifold. If x e N(A), then Ax= 0 so since A(Ax) = AO = 0, Ax is in
N(A) as well, hence N(A) is an invariant manifold.
2. R(A), the range of A, is the set of all x for which Ay = x for some y.
Clearly R(A) is a linear manifold and it is invariant since if x E R(A) then
surely Axe R(A).
which in matrix notation is (B- ,\l)a = 0 where B = ((3ji) is a k x k matrix,
and I is the k x k identity matrix. Of course, as we noted earlier, every square
matrix has at least one eigenpair, which concludes the verification .
We are now ready to prove item 4, namely that the eigenvectors of an n x n
self-adjoint matrix form an n-dimensional orthogonal basis. Suppose A = A*.
Then there is at least one eigenpair (..\IJ x 1 ) with Axi = ..\1xi and ..\1 real and
the eigenvector x 1 forms a linearly independent set (of one element). For the
induction step, suppose we have found k - 1 mutually orthogonal eigenvectors
Xi, Axi = AiXi with Ai real fori= 1, 2, ... , k- 1. We form the linear manifold
Mk = {xi (x, Xj) = 0, j = 1, 2, ... , k - 1},
called the orthogonal complement of the k- 1 orthogonal eigenvectors x1,
x2, ... , Xk-l· This manifold is invariant for A since, ifx E Mk, then (x,xj) = 0
and
(Ax,xj) = (x,Axj) = ,\j(x,x3) = 0
for j = 1, 2, ... , k- 1. Therefore, Mk contains (at least) one eigenvector Xk
corresponding to a real eigenvalue ..\k and clearly (xk, Xj) = 0 for j < k, since
Xk E Mk. The eigenvalue Ak is real since all eigenvalues of A are real. In
summary, we can state the main result of this section:
Theorem 1.5 (Spectral Decomposition Theorem) If A is ann x n self-
adjoint matrix, there is an orthogonal basis {x1, x2, ... , Xn} for which
1. Axi = AiXi with Ai real.
2. (xi,Xj) = Dij (orthogonality).
 1.3. GEOMETRICAL SIGNIFICANCE OF EIGENVALUES 17
16 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES

,.""'""''
9. The matrix Q with x; as its jth column vector is unitary, that is, ' I
I
I

q-1 = Q*. ;;
,'
I
I
I

I
I -~ b
X
4. Q* AQ =A where A is a diagonal matrix with real entries Ai· I
I

-
I I
I
I
I
I

Suppose we wish to solve Ax = b where A • = A. We now know that relative I

I
I
r- I ;'

to the basis of eigenvectors, this is a diagonal problem. That is, setting x = Qx',
I

=
I
I
I

=
b Qb' we find that Q* AQx' Ax' =b'. H all the eigenvalues are nonzero, I
I
,I

the diagonal matrix A is easily inverted for x' = A- 1 b'. Rewriting this in terms
I
I

of the original basis we find that x =QA- 1Q*b. These operations can be
I
I
I
I

summarized in the following commuting diagram. I

I
A-l I

Ax=b ~ X= A- 1 b I
I
;

x=Qx' x' =Q*x I

I
;
;

b=Qb' .t. t =
b' Q*b
I
~-
;;

A-1
Ax'= b' ~ x' = A- 1b'
The solution of Ax = b can be found directly by applying the inverse of A Figure 1.3: Graphical solution of the 2 x 2 matrix equation Ax = b with A given
(the top of the diagram) or indirectly in three steps by making a change of coor- by (1.7), using the eigenvectors x 1 and x2 of the matrix A as coordinates.
dinate system to a diagonal problem (the leftmost vertical descent), solving the
diagonal problem (the bottom of the diagram), and then changing coordinates
back to the original coordinate system (the rightmost vertical ascent).
1.3 Geometrical Significance of Eigenvalues
The geometry of diagonalization can be illustrated on a piece of graph paper. Aside from providing a natural basis in JR" in which to represent linear prob-
Suppose we wish to solve Ax = b where lems, eigenvectors and eigenvalues have significance relating to the geometry of
A= ( -~
-1 1)
1 .
(1.7) the linear transformation A.
For a real symmetric matrix A, the quadratic form q(x) = (Ax, x) produces a
real number for every vector x in JR". Since q( x) is a continuous function in IR",
We first calculate the eigenvalues and eigenvectors of A and find At = !, with it attains a maximum on the closed, bounded set of vectors x with llxll = 1.
eigenvector Suppose the maximum is attained at x = x 1 • Then for every unit vector x
Xt = ( ~) orthogonal to x 1, q(x) :::; q(xt). But, on the subset (x,x1) = 0, llxll = 1, q(x)
again attains a maximum at, say, x = x2. Continuing inductively in this way we
and A2 = -1, with eigenvector produce a set of mutually orthogonal unit vectors, x1, X2, ••• , Xn at which the
local extremal values of q(x) are attained on the set llxll = 1. We will show that
X2 = ( i)' x,
the are the eigenvectors of A and that the values q(xi) are the corresponding
eigenvalues.
We plot these eigenvectors on a piece of rectangular graph paper. Any vector
b can be represented· uniquely as a linear combination of the two basis vectorE Theorem 1.6 (Maximum Principle) If A is a real symmetric matrix and
XlJ x2, say b = a1x1 + a2x2. To find the solution x of Ax =
b (since x = q(x) = (Ax,x}, the following statements hold:
2a1x1 - a2x2) we take twice the first coordinate of b added to the negative
of the second coordinate of b with respect to the eigenvector basis. These arE 1. At = maxllzll=l q(x) = q(x1) is the largest eigenvalue of the matrix A and
depicted in Fig. 1.3. x1 is the eigenvector corresponding to eigenvalue Al·
As was suggested in the preface, this diagonalization procedure is the basif
on which most transform methods work, and is reiterated throughout this text. 2. (inductive statement) Let Ak = maxq(x) subject to the constraints
Hopefully, this commuting diagram will become emblazoned in your mind beforE
the end of this text.
a. (x,x;} = 0, j = 1, 2, ... 'k - 1,
 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES GEOMETRICAL SIGNIFICANCE OF EIGENVALUES 19
18

changed by reversing the sign of h, which is not permitted. Thus, we must have
b. llxll = 1.
(Az1 - J.'Zl, h) = 0 for all h. Taking h = Azt - J.'Xt, we conclude that
Then .X~c = q(x~c) is the kth eigenvalue of A, .Xt 2: .X2 2: · · · 2: .X~~: and x~c is
the corresponding eigenvector of A.
Azt - J.'Zl = 0,
so that Xt is the eigenvector of A. Furthermore, q(xt) = (Axt, Xt) = p(xlt Xt) =
· p, since llxtll = 1 so that max q(x) = p is the eigenvalue and the vector that
Examples: maximizes q(x) is an eigenvector of A.
To verify the inductive step, we again use Lagrange multipliers and maximize
1. Consider the matrix
A=(~ !)• P~c(x) = (.Az,x)- p((x,x) -1)-
k-1
L qJ(X,xj),
then q(x) = (Az,z) = 3vl + 2111112 + 3vt where J=l

where X1,x2, ... , Xk-1 are eigenvectors of A corresponcling to known eigenvalues

z =( t/1 )
t/2 • .X 1 2: A2 ... 2: Aie-l, and p, Ctj, j = 1, ... , k- 1 are Lagrange multipliers. Notice

We change to polar coordinates by setting 111 =

cos 6, t/2 =
sin 6 so that
=
q(6) = 3+sin26. Clearly, maxq(6) 4 occurs at 6 = 1r/4 and minq(6) 2
occurs at 6 = -'fr/4. It follows that ~1 =
4, z1 =
(~, ~)T and ~2 = 2,
.., _ ( 1 1 )T
.u2 - V'2• -V'2 • P~,:(x~c
2. In differential geometry, the curvature tensor of a surface is a syiilmeiric
2 x 2 matrix. As a result, the directions of maximal and minimal curvature
on any two-dimensional surface are always orthogonal. You can check this
out by a careful look at the shape of your forearm, for example.
To understand the geometrical significance of the maximum principle, we
view the quadtatic form q(x) = (.Az,x) as a radial map of the sphere. That is,
to each point on the sphere llxll = 1, associate q(x) with the projection of Ax
=
onto x, q(x)x (Ax, x)x. H A is positive definite (a positive definite matrix A is
one for which (Ax, x) > 0 for all x :/= 0), the surface q(x)x looks like an ellipsoid
(an American or rugby football) in JRn. (It is not exactly an ellipsoid; the level
=
surface q(x) 1 is an ellipsoid.) The maximum of q(x) occurs along the major
axis of the football-shaped object. H we intersect it with a plane orthogonal to
the major axis, then restricted to this plane, the maximum of q(x) occurs along
the semi-major axis, and so on.
To verify statement 1, we apply the method of Lagrange multipliers to q(x).
Specifically, we seek to maximize the function Pt(x) = q(x)- p((x,x) -1). H
the maximum occurs at x 11 then P 1 (x 1 +h) -Pt(xt) ~ 0 for any h. Expanding
Pt(Xt +h) we find that
Pt(Xt +h)- Pt(Xt) = 2{(Azt, h)- p(x1, h))+ (Ah, h)- p(h, h).
Notice that (Az1 , h) - p(x 1 , h) is linear in h and (Ah, h) - p(h, h) is quadratic
in h. The sign of this quadratic expression cannot be changed by changing the
sign of h. However, unless the linear expression is identically zero, its sign can
be changed by reversing the sign of h. Thus, if h is sufficiently small, and the
linear expression is not identically zero, the sign of P 1 (x1 +h)- Pt(Xt) can be
that 8?Je> = =
= 0 implies that (x,x) 1 and 8-?a~z) 0 implies that (x,XJ) = 0.
= Suppose x~,: maximizes P~.:(x) so that P~.:(x~.: + h) - P~.:(x~.:) ~ 0. Using that
(x~c,Xj)= 0 for j = 1, 2, ... , k- 1, we find that
+h)- P~,:(x~,:) = 2((.Az~,:,h}- p(x~,:,h}) + (Ah,h)- p(h,h} .
Following the same argument as before, we require (Ax~~:- pz~,:, h) = 0 for all
h. In other words, Ax~~: - J.'Xk = 0 and x~~: is an eigenvector of A. Finally,
q(x~.:) = (Az~~:,x~~:) = p(x~c,x~.:) = p since llx~~:ll = 1 so that maxq(x) is the
eigenvalue Ale and the vector that maximizes q(x) is an eigenvector of A. Clearly
Ale ~ Alc-t ~ .. · ~At· I
From the maximum principle we can find the kth eigenvalue and eigenvector
only after the previous k - 1 eigenvectors are known. It is sometimes useful to
have a characterization of Ale that makes no reference to other eigenpairs. This
is precisely the motivation of
Theorem 1.7 (Courant Minimax Principle) For any real symmetric ma-
trix A,
.X~c =min max (.Az,x),
C llzll=l
Cez:O
where C is any (k- 1) x n matrix.
The geometrical idea of this characterization is as follows. For any nontrivial
matrix C, the constraint Cx = 0 corresponds to the requirement that x lie on
some n - k + 1 dimensional hyperplane in JRn. We find A~,: by first maximizing
q(x) with llxll = 1 on the hyperplane and then we vary this hyperplane until
this maximum is as small as possible.
For example, imagine slicing a football with a plane through the origin to
get an ellipse, and finding the principal axis of the ellipse, then varying the
orientation of the slicing plane until the length of the principal axis on the slice
20 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.3. GEOMETRICAL SIGNIFICANCE OF EIGENVALUES 21

plane is minimized. The slice that minimizes the principal axis contains the two Proof: According to the minimax principle, for any quadratic form q, the
smaller eigenvectors of A, but not the largest, so we find (x2, ..X2). relative maxima are given by
Proof: To prove the minimax principle we note that since A is symmetric, there
is a unitary matrix Q that diagonalizes A, so that A = Q AQ* where A is the
..Xj =min {max q(x)},
diagonal matrix of eigenvalues ..X1 ? . X2 • • • ? ..Xn. It follows that 0 llzll=l
O:z:=O
n
where Cx = 0 represents j - 1linear constraints on x. If Xi, i = 1, 2, ... , j - 1,
(Ax,x} = (QAQ*x,x} = (Ay,y} = 'E..Xiy~,
are the vectors for which the first j - 1 relative maxima of (j(x) are attained,
i=l
then the minimum for q(x) is attained by taking the rows of C to be xi, and
where y = Q*x and Cx = 0 if and only if CQy =By= 0, where B = CQ. We (restricting x so that llxll = 1)
define p. by
~i =min {max q(x)} = Il_!ax q(x) = max q(x) ~ ..Xi+k•
0 Oz=O (z,z;)=O (z,x;)=O

p. =min { max (Ax,x}} =min {max

0 11:~:11=1
Oz=O
B 111111=1 i=1
.811=0
..XiYl}. i: since the combination of Cx = 0 and Bx = 0 places j
Bz=P

+ k- 1 constraints on
q(x). Similarly, (restricting x so that llxll = 1)
If we pick the matrix B so that By = 0 implies that Y1 = Y2 = · · · = Yk-1 = 0,
then p. :::; max11 11 11=1l:~=k ..Xiy[ = ..Xk. Now pick a different matrix B. Since B ~i =min
1/i
{ max q(x)}
(z,l/;)=0
$ max q(x) $
(z,x;}=O
max q(x) = ..Xj,
(x,z;)=O
has rank $ k - 1, the solution of By = 0 is not unique, and we can satisfy the
equation By = 0 with then- k additional restrictions YH1 = Yk+2 = · · · = where XiJ i = 1, 2, ... , j - 1 are those vectors at which the first j - 1 relative
Yn = 0. With these n- k restrictions, the augmented system, call it BaY= 0, maxima of q(x) are attained. 1
has rank$ k- 1 + (n- k) = n- 1 so always has nontrivial solutions. For this For a geometrical illustration of these results, consider a 3 x 3 positive definite
restricted set of vectors (i.e., with YH1 = Yk+2 = · · · = Yn = 0) the maximum matrix A and its associated quadratic form q(x) = (Ax,x}. If we view q(x) as
for each B might be reduced. Therefore, (restricting y so that IIYII = 1) the "radial" map of the unit sphere llxll = 1, x -t q(x)x, the surface thus
constructed is football shaped with three principal axes, the major axis, the
p. = min {max
B B11=0
t
i=l
..Xiy[ ~~
J . min { m~ ti=1 ..Xiy[}
B B.,l/=0
semi-major axis, and the minor axis with lengths . X 1 ~ . X2 ? ..Xa, respectively.
If we slice this football-shaped surface with any plane through the origin, the
intersection of q(x) with the plane is "elliptical" with a major and minor axis

? min
B
{~ax ..xk
B.,l/=0
tyr}
.
=1
= ..xk.
with lengths :X1 ?. :X2, respectively. It is an immediate consequence of the
preceding theorem that these are interlaced,

..Xt ~ ~1 ?. ..X2 ?. ~2 ~ ..Xa.

Since p.? ..Xk and p.:::; ..Xk, it must be that p. = ..Xk, as proposed. I
Using the minimax principle we can estimate the eigenvalues of the matrix Example:
A. To illustrate the usefulness of these results, suppose we wish to estimate the
eigenvalues of the symmetric matrix ·
Theorem 1.8 Suppo6e the quadratic fonn q(x) = (Ax,x} is constrained by k
linear constraints Bx = 0 to the quadratic fonn q in n - k variables. The 1 2 3 )
A= 2 3 6
relative extremal values of q, f!.enot~ ..Xt ? ..X2 ?.... · · · ? ..Xn and the relative ( 3 6 8
extremal values of q, denoted ..X1 ? . X 2 ~ · · · ? ..Xn-k satisfy the interlacing
inequality using the minimax principle. First notice that if we increase the diagonal
..Xj ~ ~j ~ ..xi+k• j = 1, 2, ... , n - k. elements of A a bit, we get the modified Illatrix

1 2 3 )
A= 2 4 6
( 3 6 9
22 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.3. GEOMETRICAL SIGNIFICANCE OF EIGENVALUES 23

which is of rank 1 and therefore has two zero eigenvalues. The range of A is
spanned by the vector (1, 2, 3)T and A has the positive eigenvalue ..\1 = 14. It
follows (see Problem 1.3.2) that the eigenvalues of A satisfy ..\1 ~ 14, ..\2 ~ 0,
..\a~ 0.
To use the minimax principle further we impose the single constraint :Ct = 0.
-
uj-1 ui uf+l
__..,

With this constraint the extremal values of q(:c) are the eigenvalues of the mj-1 mi mf+J
diagonal su bmatrix
~
A= ( 36 6)
8 . Figure 1.4: One dimensional lattice of balls connected by springs.

For this matrix we calculate that ~ 1 = 12 (with eigenvector (2, 3)T), ~2 = -1

(with eigenvector (3,-2)T), so that ..\a~ -1 ~ ..\2 ~ 0, and ..\1 ~ 12.
where u is the n-vector with components u1, u2, ... , Un and the n x n matrix A
To further improve our estimate of ..\1, observe that q(:c) = 12 for is tridiagonal with entries

a3.,3. -
- - J. -- 1, .•. , n,
kj+ki-1
mi ,

·=_,b(!)· aJJ-1
aJ,J+l
2
= km j , J = , ... ,n,
kj-1 •

= .;;;, j = 1. ... , n- 1,
We try the nearby unit vector
and all other entries are zero. Of course, one could envision a more complicated

•=;.. 0)
and find that q(:c) = ¥/ =13.07 so that 13.07 ~ ..\t ~ 14.
An improved estimate of ..\2 is found by imposing the single constraint :c2 = 0
to get the restricted quadratic form q(:c) = :c~ + 6:ct:ca + 9:c~. For this
quadratic form the extremal values are ~· It follows that -0.1098 ~
.tfB ~ ..\2 ~ 0. These estimates are actually quite good since the exact
eigenvalues are ..\ = -1, -0.076, and 13.076.
To understand further the physical significance of eigenvalues, it is useful to
think about vibrations in a lattice. By a lattice we mean a collection of objects
(balls or molecules, for example) which are connected by some restoring force
(such as springs or molecular forces). As a simple example consider a one-
dimensional lattice of balls connected by linear springs (Fig. 1.4). Let m; and
u; denote the mass and displacement from equilibrium, respectively, of the jth
ball. The equation of motion of the system is given by
cflui
ffij dt2 =k;(Uj+l-Uj)+kj-1(Uj-1-Uj)•
Here we have assumed that the restoring force of the jth spring is linearly pro-
portional to its displacement from equilibrium with constant of proportionality
ki > 0 (i.e., Hooke's law).
Suppose that the masses at the ends of the lattice are constrained to remain
=
fixed so that u 0 Un+l = 0. This system of equations can be written as
rPu =Au,
dt2
lattice for which connections are not restricted to nearest neighbor interactions
in which case the matrix A would not be tridiagonal. Something to try with
the system (1.8) is to assume that the solution has the form u = 4Jeiwt in which
case we must have
A4J = -w2 4J.
Thus, the eigenvalues >. = -w 2 of the matrix A correspond to the natural
frequencies of vibration of this lattice, and the eigenvectors 4J determine the
shape of the vibrating modes, and are called the natural modes or normal
modes of vibration for the system.
The matrix A for the simple lattice here is symmetric if and only if the
masses m; are identical. H the masses are identical we can use the minimax
principle to draw conclusions about the vibrations of the lattice.
H mi = m for j = 1, 2, ... n, we calculate that
n n
m(Au, u} = m L L aiJUiUJ
i=l j=1
n-1
= -kou~- knu;- L kj(Uj- 2
UJ+1) .
j=l
Since the constants ki are positive, A is a negative definite matrix (a negative
definite matrix is one for which (Ax,x} < 0 for x ::f: 0). Notice further that
increasing any kj decreases the quantity {Au, u} and hence, the eigenvalues of
A are decreased. From the minimax principle we have some immediate conse-
quences:
1. H the mass m is increased, all of the eigenvalues of A are increased, that
is, made less negative, so that the natural frequencies of vibration are
(1.8) decreased.
24 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 25

2. If any of the spring constants k; is increased then the eigenvalues of A The first half of the Fredholm alternative (Theorem 1.9) is easily proven. If
are decreased (made more negative) so that the natural frequencies of A*v = 0 and xo satisfies Axo = b then
vibration are increased.
(b, v) = (Axo, v) = (xo, A"v) = (xo, 0) = 0.
The natural frequencies of vibration of a lattice are also called the resonant
frequencies because, if the system is forced with a forcing function vibrating To prove the last half, suppose (v, b) = 0 for all v with A*v = 0. We write
at that frequency, the system resonates, that is, experiences large amplitude b = br + bo where br is the component of b in the range of A and bp is the
oscillations which grow (theoretically without bound) as time proceeds. This component of b orthogonal to the range of A. Then 0 = (bo, Ax) = (A*b0 , x)
observation is the main idea behind a variety of techniques used in chemistry for all x, so that A"bo = 0. Since (b, v) = 0 for all v in the null space of A*,
(such as infrared spectroscopy, electron spin resonance spectroscopy and nu- and since bo is in the null space of A*, we conclude that (b0 , b) = 0. Expanding
clear magnetic resonance spectroscopy) to identify the structure of molecules in b = br+bo we find 0 = (bo,br+bo) = (bo,br)+(bo,bo). Since (b0 ,br) = 0 it must
liquids, solids, and gases. be that bo = 0 so that b = br is in the range of A, i.e., Ax= b has a solution. 1
It is also the idea behind microwave ovens. If an object containing a water
molecule is forced with microwaves at one of its resonant frequencies, the water
molecule vibrates at larger and larger amplitudes, increasing the temperature
Example:
of the object. As you may know, microwave ovens do not have much effect on
objects whose resonant frequencies are much different than that of water, but To illustrate these two theorems in a trivial way, consider the matrix
on objects containing water, a microwave oven is extremely efficient. Microwave
ovens are set at frequencies in the 10 11 hz range to resonate with the rotational
frequencies of a water molecule. A=(!~)·
The null space of A is spanned by the vector (2, -l)T so that solutions of
1.4 Fredholm Alternative Theorem Ax = b, if they exist, are not unique. Since the null space of A • is spanned
by the vector (3, -l)T, solutions of Ax= b exist only if b is orthogonal to v
The Fredholm Alternative Theorem is arguably the most important theo- so of the form
rem used in applied mathematics (and we will see it many times in this text) as
it gives specific criteria for when solutions of linear equations exist.
b=a(!),
=
Suppose we wish to solve the matrix equation Ax b where A is an n x m which is no surprise since (1, 3)T spans the column space of A.
matrix (not necessarily square). We want to know if there is a solution, and if
so, how many solutions are possible. ·
One useful restatement of the Fredholm alternative is that the null space of
Theorem 1.9 (Fredholm Alternative Theorem) The equation Ax b has = A* is the orthogonal complement of the range of A and that together they span
= =
a solution if and only if (b, v) 0 for every vector v satisfying A • v 0. IR.m, so that IRm = R(A) EB N(A*). Stated another way, any vector b E JRm
can be written uniquely as b = br + bo where br is in the range of A and b0 is in
the null space of A•, and br is orthogonal to b0 (see Fig. 1.5).
It is interesting to note that this theorem is true for any inner product on
mn I although if one changes the inner product, the adjoint matrix A. changes

as do the vectors v for which A*v 0.= 1.5 Least Squares Solutions-Pseudo Inverses
Theorem 1.10 A solution of Ax = b (if it emts) ~ unique if and only if x = 0
Armed with only the Fredholm alternative, a mathematician is actually rather
~the only solution of Ax= 0.
dangerous. In many situations the Fredholm alternative may tell us that a
system of .equations Ax = b has no solution. But to the engineer or scientist
Proof: We prove Theorem 1.10 first. Suppose Ax = 0 for some x I- 0. If who spent lots of time and money collecting data, this is a most unsatisfactory
Axo = b then Xt = xo +ax also satisfies Ax 1 = b for any choice of a so that the answer. Surely there must be a way to "almost" solve a system.
solution is not unique. Conversely, if solutions of Ax = b are not unique then Typical examples are overdetermined systems from curve fitting. Suppose
there are vectors Xt and x2 with x = x 1 - x2 I- 0 satisfying Axt = Ax2 =b. the data points (ti, Yi), i = 1, 2, ... , n, are thought to be linearly related so that
=
Clearly Ax A(xt - x2) = Axt _: Ax2 0. = Yi = a(ti -l) + /3, where t = ~ L:~=l ti. This is a system of n linear equations
26 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 27

We have control only over the selection of x, so the minimum is attained when
we minimize II Ax - brW. Since br is in the range of A, Ax = br always has a
solution and minx IIAx- bll = llboW· We know that br is the projection of b
onto the range of A and bo is in the orthogonal complement of the range of A.
Therefore, 0 = (bo,Ax) = (A•bo,x) for all x so that A•bo = 0. Now Ax= br is
equivalent to Ax = b - bo so that Ax - b = -bo must be in the null space of
A•, which is true if and only if A• Ax= A•b. Thus the least squares solution of
Ax = b is any vector x satisfying A • Ax = A • b. One such x always exists. The
equation
A•Ax = A•b (1.10)
is called the normal equation.
Another derivation of the equation (1.10) uses ideas of calculus. Since we
wish to minimize IIAx- bll 2 =(Ax- b,Ax- b}, we let x = xo + ay, where xo
is the minimizing vector and y is an arbitrary perturbation. Since xo renders
IIAx- bll 2 a minimum, (A(xo + ay)- b,A(xo + ay)- b)~ (Axo- b,Axo- b)
for ay nonzero. Expanding this expression, we find that we must require
Figure 1.5: Graphical interpretation of the Fredholm alternative theorem. a(y, A•(Axo- b)}+ a(A.(Axo- b), y} + a 2 (Ay, Ay} ~ 0.

in the two unknowns a and /3 of the form This last expression is quadratic in a, and unless the linear terms vanish, for
sufficiently small a, a change of sign of a produces a change of sign of the entire
expression, which is not permitted. Thus we require (y, A*(Axo- b)} = 0 for
t 1 ) ( Yl ) ally. That is, we require A• Axo = A*b, which is the normal equation.
A(~)= ( t.~: ~ (~)= ~ .
t1 -
(1.9) We know from our first derivation that the normal equation always has a
solution. Another check of this is to apply the Fredholm alternative theorem
tn-t 1 Yn directly to (1.10). We require (v,A"'b} = 0 for all v with A• Av = 0. (Notice that
If there are more than two data points this system of equations does not A• A is a square, self-adjoint matrix). If A* Av = 0 then Avis simultaneously
in general have an exact solution, if only because collected data always contain in the null space of A • and in the range of A. Since these two subspaces are
error and the fit is not exact. orthogonal, Av must be 0, the only element common to both subspaces. Thus
Since we cannot solve Ax = b exactly, our goal is to find an x that minimizes (v, A•b) = (Av, b) = 0, as required.
As an example of the least squares solution, consider the curve fitting prob-
IlAx- bll. lem described earlier. We seek a least squares solution of (1.9). Premultiplying
The minimal solution is norm dependent. One natural, and common, choice
of norm is the Euclidean norm (1.4) while other choices such as llxlloo or llxll = by A*, we find (use that E~= 1 (t,- f)= 0)
E~= 1 lxsl =llxlh are useful as well, but lead to different approximation schemes. n n

Definition 1.12 For an m x n matrix A and vector b E <lim, the least squares
a '.L:(t,- f) 2
= '.L:Yi(ti- f),
i=l i=l
problem is to find x E ~ for which the Euclidean norm of Ax- b is minimized. n
A vector x (not necessarily unique) that minimizes the Euclidean norm of Ax-b n/3 = LYi!
i=l
is the least squares solution.

To find the least squares solution, recall from the Fredholm alternative that b
can always be written as b = br + bo, where br is in the range of A and bo is
orthogonal to the range of A. Since Ax - br is in the range of A then (by the
Pythagorean theorem)
IIAx- bll2;, IIAx- brll2 + llboll2·
which is always solvable for a and /3.
Other types of curve fitting are possible by changing the underlying as-
sumptions. For example, one could try a quadratic fit Yi = at~+ j3t, + 'Y· Other
choices of basis functions (such as are discussed in Chapter 2) can also be used,
but in all cases we obtain a linear least squares problem. For the exponential
fit y1 = aef3t,, the parameters a, /3 occur nonlinearly. However, the equation
28 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 29

ln Yi = ln a + /3ti is linear in ln a and /3 and leads to a linear least squares

problem.
The least squares solution always exists, however, it is unique if and only if R(A*)
= =
A* Ax= 0 has only the trivial solution x 0. H A*(Ax) 0 then AxE N(A*)
= =
and Ax E R(A) so that Ax 0. Thus, A• A is invertible if and only if Ax 0 has
no nontrivial solutions, that is, if A has linearly independent columns. H A has
linearly independent columns, then the least squares solution of Ax = b is given
N(A)
by x = (A* A)- 1 A*b, called the Moore-Penrose least squares solution.
The matrix A'= (A* A)- 1 A* is called the Moore-Penrose pseudo-inverse
of A. Notice that when A is square and invertible, A* is also invertible and
(A* A)- 1 A* = A- 1 A*- 1 A* = A- 1 so that the pseudo-inverse of A is precisely
the inverse of A. Notice also that the linear fit discussed earlier has a unique
solution if there are at least two distinct data points ti (so that the columns of X.' Ax=br
A are linearly independent).
H A does not have linearly independent columns, the least squares problem Figure 1.6: Graphical solution of the least squares problem for the matrix (1.11).
does not have a unique solution. Suppose Xp is one solution of the least squares Every vector in the manifold x : Ax = b,. minimizes IIAx- bll· The smallest
=
problem A • Ax A • b. Since A has linearly dependent columns, there are vectors such vector, denoted x', must be orthogonal to N(A).
w that satisfy Aw =0, and x =Xp + w is also a least squares solution. One
reasonable way to specify the solution uniquely is to seek the smallest possible
onto a vector in R(A) (orthogonal to N(A*)), and then find the inverse image
solution of A* Ax= A*b. To minimize llxll, we want x to satisfy (x,w} = 0 for of this vector somewhere in R(A•) (orthogonal to N(A)). (See Figure 1.6.)
all w with Aw = 0. That is, we want x to be orthogonal to the null space of A, That is, x must have the form
and therefore, in the range of A •.

Definition 1.13 The least squares pseudo-inverse of A is the matrix A'

x=a(!)
=
for which x A'b satisfies: and must satisfy
1. A* Ax = A*b.
Ax = ( :~ ) - ~( _; ) ( ( _; ) ' ( :~ )) = ~ ( ~ ~ ) ( :~ ) .
2. (x, w) = 0 for every w satisfying Aw = 0.
Since
Example: A(!)=(~).
The geometrical meaning of the pseudo-inverse can be illustrated with the it follows that

(~b1 + ~b2) A ( ! ) = ~ ( ~ ) b1 + ~ ( ~ ) b2 = ~ ( ~ ~)(~)

simple 2 x 2 example

A=(!!)· (1.11)
so that
For this matrix, 2 1 ) ( 1 )
X = ( Sbl + Sb2 1 =( 2/5
2/5
1/5 ) ( bl )
1/5 b2
R(A) = span { ( l )}• R(A*) =span { ( ~ )}' and that the pseudo-inverse of A is
I ( 2/5 1/5 )
N(A*) =span{ ( _; ) } ·
A = 2/5 1/5 .

There are a number of different numerical algorithms one can use to calcu-
The pseudo-inverse of A must project the vector late .the pseudo-inverse A'. In most situations it is not a good idea to directly
calculate (A* A)- 1 since, even if A has linearly independent columns, the calcu-
·b=(:~) lation of (A* A)- 1 is numerically poorly posed. The algorithms that we describe
30 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONs-PSEUDO INVERSES 31

below are included because they illustrate geometrical concepts, and also, in the the k vectors {uj} form an orthonormal basis for the null space of A•. Thus,
case of the last two, because of their usefulness for numerical calculations. the projection matrix P is given by P = I- 2::=
1 u,uf. Notice that if the null
space of A• is empty, then k = 0 and P =I. If the null space of A is empty we
Method 1: Gaussian Elimination row reduce the augmented matrix [A, P], but if not, we append the additional
The first idea is to mimic Gaussian elimination and find A' via ele-
constraint (w,, x} = 0 for each linearly independent w,
E N(A), and row reduce
the augmented matrix
mentary row operations. Recall that Gaussian elimination is the process of

[fr ~]·
reducing the augmented matrix [A, I] to [J,A- 1] by means of elementary row
operations. The goal of reduction is to subtract appropriate multiples of one row
from lower rows of the matrix so that only zeros occur in the column directly
below the diagonal element. For example, one step of the row reduction of the wT
I
0
matrix (equivalent to the system of equations Ax = Pb, {w 1, x) = 0, i = 1, ... , l),
au a12 B1n discarding all zero rows, which leads to [I, A'].
a22 B2n To see that this can always be done, notice that the projection P is guaran-
teed by the Fredholm alternative to project onto the span of the columns of A.
Furthermore, the rows of A span the range of A • and the vectors {Wi} span the
A= BA:A: BA:n null space of A, so that, from the Fredholm alternative JRn = R(A•) E9 N(A),
BA:+1,A: BA:+1,n the rows of A augmented with {wf} span mn
and hence can be row reduced to
an identity matrix (with possibly some leftover zero rows).
0
BnA: ann Example:
is accomplished by premultiplying A by the matrix (only nonzero elements are Consider the matrix
displayed)
1 A= ( 21 42 6)
3 .
Since A has linearly dependent columns, A has no Moore-Penrose pseudo
inverse. The null space of A is spanned by the two vectors

M.~:= I
1
-O'A:+l
-0'1:+2
1
1
w,=(_D· --=( -0·
and the null space of A • is spanned by the single unit vector

-O'n 1
u= ~ ( -~ )·
where O'J = BJA:/a,~:,~:. Specifically, the nonzero entries of M.~: are ones on the
AB a result,
diagonal and -ui in the kth column, below the diagonal. The numbers ui
are called multipliers for the row reduction. The result of a sequence of such p = I- 51 ( 1
-2
-2 ) 1 ( 4
4 = 5 2
2 )
1 .
row operations is to triangularize A, so that MnMn-1 · · · M1A = U is an upper We row reduce the augmented matrix
triangular matrix. The product of elementary row operations MnMn-1 · · · M1 =
L- 1 is a lower triangular matrix which when inverted gives A = LU. This
representation of A is called an LU decomposition. wr
A p)0
(2 4 6-
4/5
2/5 1 2 3
2/5)
1/5
0 .
By analogy, our goal is to find some way to augment the matrix A with ( wf 0 - 1 1 -1 0
2 -1 0 0 0
another matrix, say P, so that the row reduction of [A, P] yields the identity
and the pseudo-inverse [I, A']. The least squares solution we seek must satisfy Elementary row operations reduce this matrix to
Ax= br, subject to the conditions {w,,x) = 0 where {wi} spans the null space
1 0 0 2/70 1/70 )
of A, and the vector br is the projection of the vector b onto the range of A. We 0 1 0 4/70 2/70
.
can represent br as br = Pb = b.- Li=t A: {u,, b)u, = ( I- Li=t T)
A: u,u, b, where ( 0 0 1 6/70 3/70
1
32 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 33

(we have discarded one identically zero row) so that Eliminating the last row of U and last column of L (which are superfluous)
we determine that

A' = _!_
70
( ~6 3~ ) . A = LU = ( ~5 2~ ) ( 1 2 ) .
0 -2

It is easy to check that A • AA' = A •.
Method 2: L U Decomposition
Any matrix A can be written as A = LU where L is lower triangular and
U is upper triangular (with the proviso that some row interchanges may be
necessary). In practice, A is usually not reduced to a diagonal matrix, but
simply to triangular form. The decomposition is unique if we require L to
have ones on the diagonal and U to have linearly independent rows, except for
possible zero rows.
To see why the LU decomposition is desirable, suppose we want to solve
Ax= band we know that A= LU. We let Ux =
y so that Ly =b. Solving
Ly = b involves only forward substitution, and solving U x =
y involves only
backward substitution. Thus, if both L and U are full rank, the solution vector
x is easily determined.
The LU decomposition is found by direct elimination techniques. The idea
is to reduce the augmented matrix [A, I] to the form (U, L -l] by lower triangu-
lar elementary row operations. When implementing this on a computer, to save
memory, one can simply store the multipliers from the row reduction process
in the sub diagonal element that is zeroed by the row operation (see Prob-
lem 1.5.12). The sub-diagonal half of the resulting matrix is the sub-diagonal
part of L and the diagonal elements of L are all ones. If A has linearly in-
dependent rows, then zero rows of U and corresponding columns of L can be
harmlessly eliminated.
Example:
Notice that since L has linearly independent columns, the null space of A is
the same as the null space of U and similarly, since the rows of U are linearly
independent, the null space of A* is the null space of L*. The range of A is
spanned by the columns of L and the range of A • is spanned by the columns of
U*. If A = A •, then U = o:L * for some scalar o:.
Once the LU decomposition is known, the pseudo-inverse of A is easily found.
Theorem 1.11 If A = LU, then
A'= U*(UU*)- 1 (L*L)- 1 U.
This statement can be directly verified by substituting x:::: A'b into A* Ax=
A*b and using that A= LU. Since U has linearly independent rows and L has
linearly independent columns, UU* and L* L are both invertible. Notice also
that if Aw = 0, then Uw = 0 and
{w, A' b) = {w, u•wu•)- 1 (L* L)- 1L*b)
= (Uw, (UU*)- 1 (L* L)- 1L*b) = 0,
so that x is the smallest possible least squares solution. The usefulness of this
representation of A' is primarily its· geometrical interpretation.
Method 3: Orthogonal Transformations
Another important way to solve least squares problems is to transform the
matrix A to upper triangular form using carefully chosen transformations. The
advantage of transforming to a triangular system is that if the diagonal elements
are nonzero, it can be solved by backward substitution, which is not much more
difficult than solving a diagonal system. However, there is an additional bonus

n
Consider the matrix
if the matrix transformations we use are orthogonal matrices.

A=u
By row reduction, storing multipliers below the diagonal, we find that A row
Definition 1.14 A square complex matrix Q is unitary (also called orthog-
onal) if Q* = Q- 1 , i.e., Q*Q = I.
(Recall that unitary matrices first appeared in Thm. 1.5.) Unitary matri-
reduces to
ces have two important geometrical properties, namely they preserve angles

(: -~) and lengths, that is, for any vectors x and y, (Qx, Qy) = (x, Q*Qy) = (x, y),
and IIQxll = llxll. In other words, unitary transformations are rotations and
reflections.
so that
1 0 0) The transformation we seek represents A as A= QR where Q is unitary and

U=O -D· L=
( 3
5
1
2
0
1
. R is upper triangular, and this is called the QR decomposition of A. This de-
composition can always be found because this is exactly what the Gram-Schmidt
34 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 35

I
I
I
I

I
X u = x+ llxlle
u = x -llxlle I 1
I

____ ... -
I
llxll e 1 I
I llxlle e
I I
----------- Uv I
I

-Pv
Figure 1.8: Sketch of the geometry of a Householder transformation applied to
Figure 1.7: Sketch of the geometry of an elementary reflector, where Pv = f1:ii~ u the vector u = x±llxllel. The two choices of a= ±llxll give different reflections.
is the projection of v onto u.
Proof:
procedure does, namely, it represents a vector as a linear combination of pre- Since x f. -cre1, u f. 0 and U is an elementary reflector. Since cr 2 = llxll 2 ,
viously determined orthogonal vectors plus a new mutually orthogonal vector.
Here, the columns of Q are the orthogonal basis that results from applying the
llull 2 = 2a 2 + 2crxl.
Gram-Schmidt procedure to the columns of A. The matrix R indicates how to where X1 = (x, e1) is the first component of x, and
express the columns of A as linear combinations of the orthogonal basis Q.
To carry this out we could use the Gram-Schmidt procedure, however, it uu*x (x + cre1,x)u
UX = X - = X - = X - U = -CTel.
turns out that this is a numerically unstable process. Instead, we use a method U
2
+ O"Xl CT 2 + CTX1
that looks similar to Gaussian elimination, which, instead of using elementary
row operations as in Gaussian elimination, uses an object called an elementary It is wise (for numerical reasons, to avoid cancellation errors) to choose a to
reflector, or more commonly, Householder transformation. have the same sign as Xt, so that x f. -cre1 unless x = 0 already, in which case,
the transformation is not needed. I
Definition 1.15 An elementary reflector (Householder transformation) is a
The geometry of Theorem 1.12 is depicted in Fig. 1.8.
matrix of the form
We now see that we can place zeros in the first column of A, below the diag-
2uu• onal, by premultiplying A by the appropriate Householder transformation. To
U =I -llull2' provided u ':/: 0.
carry out the transformation on the entire matrix, we do not actually calculate
the matrix U but rather note that for any column vector y
It is easy to verify that U = =
u• and u•u I. The identification of the
Householder transformation as a reflector comes from the fact that if v is parallel Uy = Y _ 2uu•y = y _ 2(u,y) u,
= =
to u, then Uv -v, while if vis orthogonal to u, then Uv v. Thus, if Pv is u*u (u,u)
= =
the projection of v onto u, then v Pv +(I- P)v, and Uv -Pv +(I- P)v.
so that U y can be viewed as a linear combination of two column vectors with
In other words, U "reflects" the vector v through the plane that is orthogonal
to u. A sketch of this is shown in Fig. 1.7. a multiplier related to the inner product of the two vectors. Premultiplying
A useful property of elementary reflectors is that, as is also true of elementary by a sequence of Householder transformations, we sequentially triangularize the
row operations, they can be chosen to introduce zeros into a column vector. matrix A. We first apply the Householder transfor:rp.ation that places zeros
in the first column below the diagonal. Then we apply a second Householder
Theorem 1.12 For z E IEr', let u =
±llzll, and suppo1e z ':/: -ue1. If u = transformation that leaves the first row and first column undisturbed and places
z + ue1, then U = I - ~ u an elementary reflector whose action on z u zeros in the second column below the diagonal, and so on sequentially. Note
=
Ux -ue1 (e1 u the first natural basis vector). that if
2UkUk
uk = h- llukll 2
36 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 37

is a k x k Householder transformation that places zeros in the first column below this cannot happen, since the transformation that diagonalizes Mo is not or-
the diagonal of a k x k matrix, then the n x n matrix thogonal. However, if Mo is not symmetric, then Mn converges to a "nice"
matrix whose eigenvalues are easy to find.
In-k 0 ) Householder transformations provide a very efficient way to find the QR
V~c = ( 0 U~c decomposition of a matrix. Good software packages to do these calculations on
a computer are readily available, and a good description of the corresponding
is the Householder transformation for an n x n matrix that transforms only the theory is found in [103).
lower diagonal k x k submatrix and leaves the remaining matrix unscathed.
H Vi, V:l,· · ·, Vn are the chosen sequence of transformation matrices, then Method 4: Singular Value Decomposition {SVD)
with Q* = Vn Vn-1 · · · V1 we have
The final method by which A' can be calculated is the singular value
IIAx- bll = IIQ* Ax- Q*bll = II&- Q*bll, decomposition. This method extends the idea of the spectral decomposition
of a matrix to nonsquare matrices and is often used in applications.
where As we will see, the singular value decomposition of a matrix A is similar to
the spectral representation of a square matrix A = TAT- 1 • Once the spectral
R= ( ~)' decomposition is known (if it exists), the inverse is easily found to be A- 1 =
TA- 1T- 1 , provided the diagonal matrix A has only nonzero diagonal entries.
with R upper triangular. H the columns of A are linearly independent (the This does not, however, work to find a pseudo-inverse.
same condition as required for the Moore-Penrose pseudo-inverse) the matrix R If A is self-adjoint, then the spectral decomposition of A is A= QAQ* where
is square, has nonzero diagonal elements, and is therefore invertible. Q is orthogonal. To find the pseudo-inverse of A, we want to minimize
H we denote the vector
IIAx- bll:l = IIQAQ*x- bll = IIAQ*x- Q*bll =
Q*b= (~)I IIAy- Q*bll, (1.12)

where y = Q*x. It is an easy matter to verify (see Problem 1.5.2) that the
where the vector b1 has the same number of elements as R has rows, then pseudo-inverse of an m x n diagonal matrix D of the form
the least squares solution is found by solving Rx =b1 and for such an x,
IIAx- bll = 11~11, and this error cannot be made smaller. It follows that the
pseudo-inverse of A is given by A'= R'Q*, where R' is the pseudo-inverse of R.
D=(D0 0)
0 '
This method is most useful in the case that the matrix A has linearly inde-
pendent columns so that R is invertible and the solution of Rx = b1 is found by where D is a (square) diagonal matrix with nonzero diagonal elements is the
back substitution. H, however, R is not invertible, one must find the smallest n x m diagonal matrix
solution of 1ix = i1]., which, unfortunately, cannot be found by back substitution.
There is another important use for Householder transformations that should
D' = ( Dt
0
0)
0 .
be mentioned here. As is well known, there is no way to find the eigenvalues It follows that the pseudo-inverse of the n x n diagonal matrix A is
of a matrix larger than 4 x 4 with a finite step algorithm. The only hope for
larger matrices is an iterative procedure, and one of the best is based on the
Q R decomposition.
A'= ( x-l0 o)
0 '
Two matrices have the same eigenvalues if they are related through a simi-
larity transformation. The goal of this algorithm is to find an infinite sequence where A is the nonzero part of A. Therefore we take
of similarity transformations that converts a matrix Mo into its diagonal matrix
of eigenvalues. y = Q*x = A'Q*b
The method is to use Householder transformations to decompose the matrix andy= Q*x is the smallest minimizer of (1.12). However,
Mn into the product of an orthogonal matrix and an upper triangular matrix
Mn = QnRn· To make this into a similarity transformation we reverse the order X= QA'Q*b
of matrix multiplication and form Mn+l = RnQn so that Mn+l = Q;; 1 MnQn.
Now the amazing fact is that if M 0 is a real symmetric matrix, then Mn con- has the same norm as Q*x and so is the smallest least squares solution of (1.12).
verges to the diagonal matrix of eigenvalues. Of course, if Mo is not symmetric, It follows that the pseudo-inverse of A is A'= QA'Q*.
38 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 39

The geometrical interpretation of this pseudo-inverse is as follows. H A = Suppose A* A bas k positive eigenvalues A1, A2 1 ••• , Ak > 0, and Ak+l =
QAQ*, then Q represents the basis relative to which A has a diagonal represen- ).k+ 2= · · · =An = 0. Let Yi = $t.
i = 1, 2, ... , k, and notice that
tation. H one of the diagonal elements of A is zero, then of course, the inverse
of A does not exist. To find A'b, we form Q*b which expresses b relative to the (y;,yi} = .JX~\f>;;"(Ax;,Axi} = .JXIJ>:;(xi,A'"Axi}
new coordinate system. The expression A'Q*b projects Q*b onto the range of
A and determines the smallest element whose image under A is Q* b. Finally = .:§f(x;,xi} = 5ii·
Q(A'Q*b) expresses this vector in terms of the original coordinate system (see
Problem 1.5.6). Since y; E mm I this guarantees that k :::; m. We can use the Gram-Schmidt
The change of coordinates Q identifies both the range and null space of A. procedure to extend the y;'s to form an orthonormal basis for !Rm, {y;}~ 1 . Let
What we have done here is to project the vector b onto the range of A and U be the matrix with columns Yl, Y2, ... , Ym. The vectors y; are the orthonormal
then find another vector in the range of A whose image is this projection. Since eigenvectors of AA •. We now examine the entries of U* AV. For i ::; k, the ijth
A* =A, the matrix Q is appropriate for both of these transformations. entry of u• AV is
This construction of A' can work only for symmetric matrices where R( A •) =
R(A). Unfortunately, this construction fails for nonsymmetric matrices for two
(y;,AxJ}= -it-(Ax;,Axj}
'S.~'
= +(x;,A*Axi}
v Ai

important reasons (see Problem 1.5.9}. First, since A* :f. A, R(A*) :f. R(A) so
projecting onto R(A) does not also put us on R(A*). Furthermore, if A* :f. A,
the basis provided by the diagonalization of A is not an orthogonal basis and
hence projections are not orthogonal.
What is needed is an analogous construction for nonsymmetric matrices.
This construction is provided by the singular value decomposition, which we
now state.
Theorem 1.13 Suppose A is any m x n matrix.
1. A can be factored as
A= UEV*, (1.13}
where U (m x m) and V (n x n) are unitary matrices that diagonalize AA•
and A • A respectively, and E is an m x n diagonal matrix {called the
= ~(Ax;,Axi) = .J>:iaiJ·
Fori> k, note that AA*y; = 0 so that A*y; is simultaneously in the null space
of A and the range of A*. Therefore (the Fredholm alternative again}, A*y; = 0
and (y;,Axi} = (A*y;,xi} = 0 fori> k. It follows that U*AV = E where
= =
!: (O';j), O'ij = .J>:i5;i, i 1,2, ... , m, j =
1, 2, ... ,n, and this is equivalent
to (1.13).
It is noteworthy that the orthogonal vectors {x 1 , x 2 , ••• , xk} span the range
of A • and the vectors {x k+l, ... , Xn} span the null space of A. Similarly, the
vectors {y1, Y2, ... Yk} span the range of A and {Yk+l• ... , Ym} span the null
space of A*. Thus the orthogonal matrices U and V provide an orthogonal
decomposition of !Rm and !Rn into !Rm = R(A) EB N(A*) and !Rn = R(A"') EB
N(A), respectively. Furthermore, the singular value decomposition A= UEV*
shows the geometry of the transformation of X by A. In words, v· X decomposes

= =
matrix of singular values) with diagonal entries O'ii ...;>:i1 where Ai 1 i x into its orthogonal projections onto R(A•) and N(A), then EV*x rescales the
1,2, ... ,min(m,n), are the eigenvalues of AA* {see Problem 1.B.3}. projection of x onto R(A*), discarding the projection of x onto N(A). Finally,
multiplication by U places !:V*x back onto the range of A.
B. The pseudo-inverse of A is It is now clear that to minimize
A'= V!:'U., (1.14) IlAx- bll 2 = IIU!:V*x- bll 2 ,
where !:' 1 the pseudo-inverse of !: 1 is an n x m diagonal matrix with we take
diagonal entries o11 = 1II =-it-
-(1'
V AI
provided O'ii :f. 0 and 0 otherwise. X= V!:'U*b
so that the pseudo-inverse of A is as stated in (1.14). I
The factorization A = U!:V* is called the singular value decomposition of The singular value decomposition gives a nice geometrical interpretation for
A. the action of A. That is, first there is a rotation by V*, followed by rescaling
Proof: To show that this decomposition always exists, notice that the matrix by E and finally a second rotation by U. Similarly, to find the least squares
A• A is a symmetric n x n matrix with non-negative eigenvalues A1, A2 1 ••• , An· solution of Ax = b, we first use U to decompose b into its projection onto R(A)
Let V be the n x n matrix that diagonalizes A• A. The columns of V are the and N(A*). The component on R(A) is rescaled by E' and then transformed
orthogonal eigenvectors x1,x2, .•. ,xn of A* A, normalized to have length one. onto R(A*) by V.
Then This again illustrates nicely the importance of transform methods. As we
A* A= VAV*. know, the least squares solution of Ax = b can be found by direct techniques.
40 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 41

However, the change of coordinates provided by the orthogonal matrices U, V in the direction orthogonal to the nearly parallel straight lines of the original
transforms the problem Ax= b into ~y = b where y = V'"x, b = U'"b, which is problem. In other words, the singular value decomposition identified the stable
a diagonal (separated) problem. This least squares problem is easily solved and and unstable directions for this problem, and by purposely replacing € by zero,
transforming back we find, of course, x =
V~'U'"b. This transform process is we were able to keep the unstable direction from contaminating the solution.
illustrated in the commuting diagram In a more general setting, one can use the singular value decomposition to
filter noise (roundoff error) and stabilize inherently unstable calculations. This
A'
Ax=b --t x=A'b is done by examining the singular values of A and determining which of the
y=V'"x x=Vy singular values are undesirable, and then setting these to zero. The resulting
b=U'"b
.t. t b=Ub pseudo-inverse does not carry with it the destabilizing effects of the noise and
E' roundoff error. It is this feature of the singular value decomposition that makes
~y=b ~ y= ~'b
it the method of choice whenever there is a chance that a system of equations
where the top line represents the direct solution process, while the two verti- has some potential instability.
cal lines represent transformation into and out of the appropriate coordinate The singular value decomposition is always recommended for curve fitting.
representation, and the bottom line represents solution of a diagonal system. The SVD compensates for noise and roundoff error prppagation as well as the
The singular value decomposition also has the important feature that it fact that solutions may be either overdetermined or underdetermined. In a
allows one to stabilize the inversion of unstable (or ill-conditioned) matrices. least squares fitting problem it is often the case that there is simply not enough
To illustrate this fact, consider the matrix experimental evidence to distinguish between certain basis functions and the
resulting least squares matrix is (nearly) underdetermined. The SVD signals
A=(1+-Jfo
2-710 2+-fto
1-1fo), this deficiency by having correspondingly small singular values. The user then
has the option of setting to zero small singular values. The decision of how
small is small is always up to the user. However, with a little experimentation
where € is a small number. We can visualize what is happening by viewing the one can usually learn what works well.
equation Ax = b as describing the intersection of two straight lines in JR 2 by
noting that for € small the lines are nearly parallel. Certainly small changes in
b give large changes to the location of the solution. Thus, it is apparent that 1.5.1 The Problem of Procrustes
the inversion of A is sensitive to numerical error. The Householder transformation solves the problem of finding an orthogonal
We calculate the singular value decomposition of A to be transformation that transforms one vector into another, provided the lengths of

- v'10 (12 -12)(v'100

the two vectors are the same. A generalization of this problem, called the prob-
A--1 0)(1 1) lem of Procrustes3 is to seek an orthogonal transformation that transforms
€ 1 -1 '
one set of vectors into another, as best possible.
and the inverse of A to be Suppose we have two sets of vectors in JRn, represented as the columns of two
n x k matrices A and B, and we wish to find the best orthogonal transformation
A-1 --L(1 Q that aligns A with B. That is, we wish to minimize
- v'IO 1 -11)(fto0 0)(1
€ 2 -12)
=( 0.1 + eJro 0.2 - e{to ) . IIQA-BII 2 •
0.1 - eJrn 0.2 + e:;/i6 Before we can solve this problem we must define an appropriate norm for
matrices. A useful matrix norm for this problem is the Frobenius norm
It is clear that when the singular value € is small, the matrix A- 1
is very large
=
and the solution x A- 1 b is unstable. IIAII~ = Tr(A* A) =Tr(AA*).
If instead of using the full inverse, we replace € by zero and use the singular
value decomposition to find the pseudo-inverse, (see Problem 1.5.14). In terms of this norm, the problem is to minimize

A' _ _
1 ( 1 1) ( 'f0to 0) ( 1 2 ) _ 1:_ ( 1 2 )
IIQA- Bll~ = Tr(AA'") + Tr(BB'")- 2Tr(QAB*).
- v'iO 1 -1 0 2 -1 - 10 1 2 ' 3
Procrustes was a legendary Greek outlaw, living in Athens, who would offer his hospitality
to travelers, and then force them to lie on an iron bed, stretching them on a rack if they were
the solution is now stable. That is, small changes in the vector b translate into too short or lopping off limbs if they were too long for the bed - an unusual method of
only small changes in the solution vector x =
A'b and, in fact, x varies only coordinate transformation,--to be sure.
42 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES ).6. APPLICATIONS OF EIGENVALUES AND EIGENFUNCTIONS 43

Thus, it is equivalent to maximizing Tr(QAB*). where eAt is the matrix whose only nonzero components are the diagonal ele-
Now suppose that the matrix AB* has the singular value decomposition ments e).it, j = 1, 2, ... , n. It follows that

AB* =U~V*. x(t) = T- 1eAtTx(O}

Then, so that
eAt ~ T-leAtT.

Tr(QAB*) = Tr(QU~V*) = Tr(V*QU~) = 'Eziiqi $ L:ai,

i i 1.6.2 The Power Method and Positive Matrices
where Z = V*QU. This inequality is valid since the diagonal elements of a There are many situations where the only eigenvalue of interest is the one with
real unitary matrix cannot be larger than one, and the singular values ai are largest magnitude. It is easy to calculate the largest eigenvalue and correspond-
non-negative. However, it is also clear that equality is obtained if and only if ing eigenvector using a method called the power method. The power method
=
Z I, in which case involves "powering" a matrix. We start with some initial vector x 0 , and then
Q=VU*. form the sequence of vectors
XI;
X1;+1 = Allx~;ll" (1.16}
1.6 Applications of Eigenvalues and Eigenfunc-
Theorem 1.14 If the largest (in magnitude) eigenualue At of A is unique with
tions algebraic multiplicity one, and has a one-dimensional eigenspace spanned by
It is not possible to overstate the importance of eigenvalues and eigenfunctions in
ifJ,., and if (:eo, l/11) :f: 0, lll/Jtll = 1, then
applications. In this section, we give a short introduction to a few of the ways lim XI; = AtlPl·
in which an understanding of eigenvalues and eigenfunctions gives important k-+oo
insight or capability in applied problems.

1.6.1 Exponentiation of Matrices Proof: We suppose that At is the largest eigenvalue of A and that t/11 is the
corresponding eigenvector of A •, (l/Jt. tPt) = 1. We represent the initial vector
=
The solution of the scalar ordinary differential equation ~ au with u(O) Uo = as xo = a1l/Jt + r1 with a1 = (xo, t/11) and note that rt is orthogonal to tPI.
=
is u(t) uae'". It would be nice if we could make a similar statement for the (rt, tPt) = 0. The orthogonal complement of tPt is an invariant subspace of A
vector differential equation = = = =
since (Ar, t/Jt) (r, A*t/Jt) (r, At tPt) 0 if (r, tPt) 0. Furthermore,
dx=Ax (1.15)
dt IIArll $ Kllrll
where A is ann x n matrix and x(t) is a vector in m.n.
The problem, however, for some constant K < IAtl, whenever (r,t/Jt) = 0.
is that we do not know how to interpret the matrix eAt. Now we calculate that
The answer is readily found if the matrix A can be diagonalized. If there is
a similarity transformation so that A= T-tAT, then (1.15) becomes Akxo = a1Atl/J1 + r~;
dx where
T-
dt
=T AT-1Tx = ATx llr~;ll = IIAAiroll $ Kkllroll,
so that
so that
dy
-=Ay
lim ,1
k-+oo "l
~;A"xo-+ atlPl·
dt
Thus, if a1:f: 0, our result is established. Furthermore, the rate of convergence
where Tx = y. Since A is a diagonal matrix, the solution of the system !!If = Ay is approximately f, which is a measure of the separation between the largest
is readily written as eigenvalue At and tillother eigenvalues of A. 1
y(t) = eAty(O),
 44 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES
Example: The Ranking of Teams
Every Fall a controversy rages throughout the United States trying to decide
which is the best collegiate football team {division I-A). The dilemma is that
there are over a hundred teams vying for the title, but no team plays more
than 12 or 13 opponents, so there are often good teams that have not been
matched up head to head. It is certain that this same controversy occurs in
many other sports and in many other countries.
A possible resolution to this controversy is to devise a ranking scheme that
assigns a rank to each team based on its performance. One idea (there are
several) for how to calculate a team's rank is as follows: We assume that
team i has rank r;, a number between 0 and 1. The team earns a score based
on its games with other teams
8; = _!_ L a;Jrh
n; J'f.i
where a;J is a number that reflects the result of the head to head contest
between teams i and j. The score depends on the product of a;; and r; in
order to give some weight to the strength of opposition. It is reasonable to
take a;i + ai; = 1 with a;J > aii if team i beat team j in their head to head
contest. The division by n;, the total number of games played by team i, is
necessary to normalize the score 8;. The entry a;; is zero if team i has not
played team j, and of course, a;t = 0.
It is now reasonable to suppose that the score and the rank of a team should
be related by ..\r, = s;, so that the determination of rank reduces to an
eigenvalue problem Ar = ..\r, where the entries of A are the weighted elements
a;; /n;. The question is to determine if this equation has a useful solution.
This leads us to consider an important class of matrices called positive ma-
trices. A positive matrix is a matrix A all of whose entries are positive. Sim-
ilarly, a nonnegative matrix is one all of whose elements are nonnegative. The
matrix that we constructed to rank football teams is nonnegative. The classi-
cally important statement about these matrices is called the Perron Frobenius
Theorem.
Theorem 1.15 (Perron Frobenius) A nonnegative matrix A has a nonneg-
ative eigenfunction with corresponding positive eigenvalue. Furthermore, if the
matrix is irreducible, the eigenfunction is unique and strictly positive, and the
corresponding eigenvalue is the largest of all eigenvalues of the matrix.
We leave the proof of this theorem to the interested reader, which can be
, found in several sources, including [56]. There are several equivalent ways to de-
scribe irreducibility. Perhaps the most geometrical is that a matrix is irreducible
if for every pair of integers i, j, there is a sequence k1 , k2 , ••• , km for which the
product Oik 1 OA: 1 A: 2 ••• akmi is nonzero. In practical terms for football teams, this
means that between any two teams there is a path of common competitors.
The implementation of the Perron Frobenius theorem uses the power method.
There is a positive number, say p.,_ so that the eigenvalues of the matrix A+ p.I
1.6. APPLICATIONS OF EIGENVALUES AND EIGENFUNCTIONS 45
are all strictly positive. FUrthermore, the largest eigenvalue of the matrix A+ p.I
is ,\ + p., where tjJ is the positive eigenvector and ,\ the corresponding positive
eigenvalue of A. Hence, if we power the matrix A + JLl, convergence to tjJ is
assured (provided A is irreducible), and tjJ is the positive ranking vector we
seek.
1.6.3 Iteration Methods
The power method is one of the many iteration methods that allow rapid extrac-
tion of important information about a matrix. In fact, matrix iteration is the
only practical method by which to solve many important problems associated
with partial differential equations. Direct methods, such as Gaussian eliminar-
tion, are certain to work if the problem is small enough, but direct methods
have serious difficulties when matrices are large and sparse (a sparse matrix is
one which has mostly zero entries), as they often are with the simulation of
partial differential equations.
The main idea of many iteration methods is to split the matrix A into two
parts, A = A1 + A2, where A1 is invertible and easily solved, and then to write
the equation Ax = b as
A1x = b- A2x.
We then define an iterative procedure by
AlXHl = b- A2Xk· (1.17)
The convergence of this iteration is easily seen to depend on the eigenvalues
of the matrix A1 1 A2 . That is, if xis a solution of the problem Ax= b, then
Xk+l - x = A1 1 A2(xk - x).
If A1 1 A2 has eigenvectors t/J; with corresponding eigenvalues J.l.j, and if Xo -x =
Ej=l a;t/J;, then
Xk- X=~ OjJ.l.Nj·
j
The iteration converges provided the largest eigenvalue of A1 1 A2 is smaller than
1 in absolute value.
Now the trick is to split A in such a way as to make the largest eigenvalue
of A1 1 A2 as small as possible, thereby making convergence as fast as possible.
The easiest split of A is perhaps
A=D+L+U
where D is the diagonal part of A, and L and U are lower and upper triangular
(off-diagonal) parts of A, respectively. Then, if we choose A1 = D, and A2 =
L + U, iterates are called Jacobi iterates. These iterates are certainly easy to
calculate, since they only involve solving the system
Dxn+l = b- (L + U)xn,
46 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.6. APPLICATIONS OF EIGENVALUES AND EIGENFUNCTIONS 47

which, if the diagonal elements are all nonzero, is easy and fast. However, it is Now suppose that !Jmsn and /Jmaz are the smallest and largest eigenvalues of
not much harder to take At= D + U and A 2 = L, in which case the iterates A1 1 A 2 ,
and that -1 < l'min < l'maz < 1. Then the optimal relaxation pa-
rameter is found by balancing the numbers 1- w(JJj + 1) between -1 and 1,
(D + L)xn+l = b- Uxn taking
2
require solution of a lower triangular system of equations at each step. This Wopt = 2 + Jlomln + Jloma3l 1

method goes by the name of Gauss-Seidel iterates.

in which case the eigenvalue that determines the rate of convergence for the
The key determinant to the success of these methods is the size of the eigen-
SOR method is
values of A} 1 A2 • While these cannot be calculated easily for many matrices, it /Jmaz - Jlomln
is illustrative to see what these are for some simple examples. ..\soR = 2 + l'maz + Jlomln
Suppose D is a scalar multiple of the identity matrix, D = dl. H t/J is an
eigenvector of A with eigenvalue ..\, At!J =
..\t/J, then t/J is also an eigenvector of This is an improvement unless JLmsn = -l'maz·
D- 1 (L + U). That is,
Multigrid Methods
n- 1
(£ + U)t/J = (~ -1)€/J. The solution of large systems of equations which come from the discretization
of partial differential equations has been revolutionized with the discovery of a
It is an immediate consequence that the Jacobi method converges whenever all new approximate solution technique called the multigrid method. As we will
eigenvalues ..\ of A satisfy 0 < ~ < 2. Thus, for example, if A is diagonally see, the success of this method is completely dependent upon its relationship to
dominant (a matrix A= (atJ) for which IBid~ L#t latJD , Jacobi iterates are the eigenvalues and eigenfunctions of the underlying problem.
convergent. However, at this point in our development of transform theory, it is not
It is always possible to try to accelerate the convergence of an iterative yet appropriate to discuss this important solution technique, so we defer the
method. Acceleration is the goal of discussion of multigrid methods to Chapter 8.

Atz~+l = b- A2Zn 1 Zn+l = wx~+l + (1 - w)xn

for some parameter w. Notice that if w =
1, the iterations are unchanged.
With w i- 1, this method is called successive over-relaxation or SOR, and
the parameter w, usually taken to be bigger than 1, is called the relaxation
parameter. Notice that if the relaxation parameter is less than one then the new
guess Zn+t is an interpolation between z~+l and Zn, whereas if the relaxation
parameter is greater than one, the new guess is an extrapolation.
It is known that for many matrices the Jacobi method converges more slowly
than the Gauss-Seidel method, and with careful choice of the relaxation param-
eter, the SOR modification of Gauss-Seidel iterates converges fastest of them
all. It is not hard to see how to choose the relaxation parameter to get improved
convergence from the SOR method.
Suppose the eigenvalues of A}1 A2 are known, with
1
Ai A2tPJ = l'itPi·
Further Reading
There are numerous linear algebra books that discuss the introductory material
of this chapter. Three that are recommended are:
• P. R. Halmos, Finite Dimensional Vector Spaces, Van Nostrand Reinhold,
New York, 1958.
• G. Strang, Linear Algebra and its Applications, 3rd ed., Academic Preas,
New York, 1988.
• R. A. Horn and C. R. Johnson, MatriX Analysis, Cambridge University
Press, Cambridge, 1985.
The geometrical and physical meaning of eigenvalues is discussed at length in
the classic books

The convergence of the SOR method is assured if the eigenvalues of • R. Courant and D. Hilbert, Methods of Mathematical Physics, Volume I,
Wiley-Interscience, New York, 1953.
A} 1 ( -wA2 + {1 ·~ w)A1 ) = -wA} 1 A2 + (1- w)I
• J. H. Wilkinson, The Algebraic Eigenvalue Problem, Oxford University
1 Preas, London, 1965.
are less than one in magnitude. However, if tPi is an eigenvector of Ai A2, then
Numerical aspects of linear algebra are discussed, for example, in
( -wAi1 A2 + (1- w)I)t!JJ = (1- w(JJJ + 1))tPi·
48 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES PROBLEMS FOR CHAPTER 1 49

• G. W. Stewart, Introduction to Matrix Computations, Academic Press, Problems for Chapter 1

New York, 1980,
Problem Section 1.1
• G. E. Forsythe, M.A. Malcolm and C. B. Moler, Computer Methods for
1. Prove that every basis in a finite dimensional space has the same number
Mathematical Computations, Prentice-Hall, 1977,
of elements.
• G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed., Johns 2. Show that in any inner product space
Hopkins University Press, Baltimore, 1996,
2 2
llx+vll + llx -vll = 2llxW + 2llvW.
• L. N. 'frefethen and D. Bau, Numerical Linear Algebra, SIAM, Philadel-
phia, 1997, Interpret this geometrically in JR?.
and for least squares problems in 3. (a) Verify that in an inner product space,

• C. L. Lawson and R. J. Hanson, Solving Least Squares Problems, SIAM, 1

Philadelphia, 1995,
Re (x, y} = 4(11x + vW -llx- vll 2 ).
while many usable programs are described in (b) Show that in any real inner product space there is at most one inner
product which generates the same induced norm.
• W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling, Nu-
(c) In IRn with n > 1, show that llxiiP = CE~= 1 IxkiP) 11P can be induced
merical Recipes 1n .Fbrtran 77: The Art of Computing, Cambridge Uni- · by an inner product if and only if p = 2.
versity Press, Cambridge, 1992.
4. Suppose f(x) and g(x) are continuous real valued function defined for
This book (in its many editions and versions) is the best selling mathematics x E [0, 1). Define vectors in IRn, F = (f(xt), j(x2), ... , f(xn)) and G =
book of all time. (g(xt), g(x2), ... , g(xn)), where Xk = kjn. Why is
The best and easiest way to do numerical computations for matrices is with
Matlab. Learn how to use Matlabl For example, 1 n
(F, G}n = nL f(xk)g(xk)
• D. Hanselman and B. Littlefield, Mastering Matlab, Prentice-Hall, Upper k=l
Saddle River, NJ, 1996.
with Xk = ~'not an inner product for the space of continuous functions?
While you are at it, you should also learn Maple or Mathematica.
5. Show that
• B. W. Char, K. 0. Geddes, G. H. Gonnet, B. L. Leong, M. B. Monagan,
and S. M. Watt, Maple V Language Reference Manual, Springer-Verlag,
(!,g)= 1 1
(!(x)g(x) + f'(x)g'(x)) dx.
New York, 1991, is an inner product for continuously differentiable functions on the interval
[0, 1).
• A. Heck, Introduction to Maple, 2nd ed., Springer-Verlag, New York, 1996.
6. Show that any set of mutually orthogonal vectors is linearly independent.
• S. Wolfram, Matbematica, 3rd ed., Addison-Wesley, Reading, MA, 1996.
7. (a) Show that IRn with the supremum norm llxlloo = maxk{lxkl} is a
Finally, the ranking of football teams using a variety of matrix algorithms normed linear vector space.
is summarized in (b) Show that IRn with norm llxl It = :E~=l lxk I is a normed linear vector.
• J.P. Keener, The Perron Frobenius Theorem and the ranking of football 8. Verify that the choice-y =~minimizes llx--yyll 2 • Show that l(x, y}l 2 =
teams, SIAM Rev., 35, 80-93, 1993. llxll 2 ·llvll 2 if and only if x andy are linearly dependent.
50 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES pROBLEMS FOR CHAPTER 1 51

9. A Starting with the set {1, x, x 2 , ••• , xk, ... }, use the Gram-Schmidt pro- (c) Is the converse true?
cedure and the inner product
3. (a) Show that if A is ann x m matrix and B is an m x n matrix, then
(f,g} = ib f(x)g(x)w(x)dx, w(x) > 0
AB and BA have the same nonzero eigenvalues.
(b) Show that the eigenvalues of AA • are real and non-negative.
to find the first five orthogonal polynomials when 4. Show that the eigenvalues of a real skew-symmetric (A = -AT) matrix
are imaginary.
(a) a= -1, b = 1, w(x) = 1 (Legendre polynomials)
(b) a= -1, b = 1, w(x) = (1- x 2 )- 112 (Chebyshev polynomials) 5. Find a basis for the range and null space of the following matrices:
(c) a= 0, b = oo, w(x) = e-~ (Laguerre polynomials) (a)
a= -oo, b = oo, w(x) = e-:~~ (Hermite polynomials)
2
(d)
Remark: All of these polynomials are known by Maple.
10. ,!;!. Starting with the set {1, x, x2 , ••• , xn, ... } use the Gram-Schmidt pro-
cedure and the inner product
A~o D·
(b)
1
(f,g} = /_ (f(x)g(x) + f'(x)g'(x))dx 1 2 3)
1

to find the first five orthogonal polynomials.

A=
( 3 1 2
1 1 1
.

Problem Section 1.2

6. Find an invertible matrix T and a diagonal matrix A so that A = TAT- 1
for each of the following matrices A:
1. (a) Represent the transformation whose matrix representation with re-
(a)
spect to the natural basis is

( 1/41 1/40 1/20 )

A=
( 112)
2 1 3
1 0 1 0 0 1

relative to the basis {(1, 1, O)T, (0, 1, l)T, (1, 0, l)T}. (b)
(b) The representation of a transformation with respect to the basis
{(1,1,2}T,(1,2,3)T,(3,4,1)T} is ( -~ ~)

c00)
1 1 1) (c)
A= 2 1 3 .
( 1 0 1
1 2 0
Find the representation of this transformation with respect to the
2 1 3
basis {(1, 0, O)T, (0, 1, -l)T, (0, 1, 1)T}.

c10)
2. (a) Prove that two symmetric matrices are equivalent if and only if they (d)
have the same eigenvalues {with the same multiplicities).
(b) Show that if A and B are equivalent, then 0 1 0
detA = detB. 0 1 1
52 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES pROBLEMS FOR CHAPTER 1 53

(e) Problem Section 1.3

1. Use the minimax principle to show that the matrix
1/2 1/2 ~/6 )
1/2 1/2 ~/6 .
( ~/6 ~/6 5/6 2 4 5 1 )
4 2 1 3
7. Find the spectral representation of the matrix ( 5 1 60 12
1 3 12 48
A= ( _; ~). has an eigenvalue~ < -2.1 and an eigenvalue At > 67.4.
=
illustrate how Ax b can be solved geometrically using the appropriately 2. (a) Prove an inequality relating the eigenvalues of a symmetric matrix
cliosen coordinate system on a piece of graph paper. before and after one of its diagonal elements is increased.
8. Suppose P is the matrix that projects (orthogonally) any vector onto a (b) Use this inequality and the minimax principle to show that the small-
manifold M. Find all eigenvalues and eigenvectors of P. est eigenvalue of
84 48 -44)
9. The sets of vectors {1/>i}f.:tJ {,P,}f::t are said to be biorthogonal if
(1/>,,P;} = 6,;. Suppose {1/>slr:t and {,P,}f:t are biorthogonal.
A=
(4 -4 3
(a) Show that {1/>i}f::t and {1/li}f::t each form a linearly independent set. is smaller than -1/3.
(b) Show that any vector in JRn can be written as a linear combination
· of {t/>i} as
n
x= :Eait/>i 1 23)
where a,
= (x, 1/Js)·
i=t ( 2 2 4
3 4 3

(c) Express (b) in matrix form; that is, show that is not positive.
n
4. The moment of inertia of any solid object about an axis along the unit
x= LP,x vector x is defined by
i=t

where P1 are projection matrices with the properties that P( =

P,
J(x) = L~(y)pdV,
=
and P,P; 0 fori¥:. j. Express the matrix Pi in terms of the vectors where d 11 (y) is the perpendicular distance from the point y to the axis
t/>1 and ,p,. along x, p is the density of the material, and R is the region occupied by
10. (a) Suppose the eigenvalues of A all have algebraic multiplicity one. the object. Show that J(x) is a quadratic function of x, J(x) = :z;T Ax
Show that the eigenvectors of A and the eigenvectors of A• form where A is a symmetric 3 x 3 matrix.
a biorthogonal set. 5. Suppose A is a symmetric matrix with eigenvalues At ~ ..\2 > As ~ •.••
(b) Suppose and A•,p, = >..,,p,, i = 1, 2, ... , n and that
At/>i = Ast/>i Show that
Ai ¥:. A; for i ¥:. j. Prove that A = E~1 A,P, where P, = tf>,,p; is a max(u,v)=O(Au, u} + (Av, v} = At + ..\2
projection matrix. Remark: This is an alternate way to express the
spectral decomposition theorem for a matrix A. where !lull = llvll = 1.
(c) Express the matrices C and c-t, where A= CAc-t, in terms of t/>1
and ¢ 1• Problem Section 1.4
=
(d) Suppose At/> At/> and A,P = X,p and the geometric multiplicity of A 1. Under what conditions do the matrices of Problem 1.2.5 have solutions
is one. Show that it is. not necessary that (I/>, ,P} ¥:. 0. Ax = b? Are they unique?
54 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES pROBLEMS FOR CHAPTER 1 55

2. Suppose P projects vectors in JRn (orthogonally) onto a linear manifold 3. (a) For any two vectors x,y E mn with llxll = IIYII find the Householder
M. What is the solvability condition for the equation Px = b? (orthogonal) transformation U that satisfies Ux = y.
3. Show that the matrix A= (lliJ) where a,i = (</Jit</JJ) is invertible if and (b) Verify that a Householder transformation U satisfies U*U =I.
only if the vectors <Pi are linearly independent.
4. Use the Gram-Schmidt procedure (even though Householder transforma-
4. A square matrix A (with real entries) is positive-definite if (Ax, x) > 0 for tions are generally preferable) to find the QR representation of the matrix
all x :/: 0. Use the Fredholm alternative to prove that a positive definite
matrix is invertible.
A= 2 l 3)
Problem Section 1.5
(9 1 2
4 2 1 .

1. Use any of the algorithms in the text to find the least squares pseudo- 5. For the matrix

( 5 -3)
inverse for the following matrices:
A=

c00)
(a) 0 4 I

illustrate on a piece of graph paper how the singular value decomposition

0 1 0 A = UEV* transforms a vector x onto Ax. Compare this with how
1 1 0
A= TAT- 1 transforms a vector x onto Ax.
6. For the matrix
(b)
A=(~!),
( ~ -~)
-1 3
illustrate on a piece of graph paper how the least squares pseudo-inverse
A' = Q- 1 A'Q transforms a vector b into the least squares solution of
Ax=b.
(c)
7. For each of the matrices in Problem 1.2.6, \lSe the QR algorithm to
( 3 1 2) form the iterates An+l = Q;; 1 AnQn, where An = QnRn· Examine a few
-1 1 -2 of the iterates to determine why the iteration works and what it converges
to.
(d)
8. For the matrices
( -1 0 1)
-1 2/3 2/3 . A=(~~)
-1 -2/3 7/3
and
A ( 1.002 0.998 )
(e) Jl. The linear algebra package of Maple has procedures to calculate = 1.999 2.001 I

the range, null space, etc. of a matrix and to augment a matrix with
another. Use these features of Maple to develop a program that uses
exact computations to find A' using the Gaussian elimination method
(Method 1), and use this program to find A' for each of the above
matrices.
illustrate on a piece of graph paper how the least squares pseudo-inverse
A' = VE'U* transforms a vector b onto the least squares solution of
Ax = b. For the second of these matrices, show how setting the smallest
singular value to zero stabilizes the inversion process.

2. Verify that the least squares pseudo-inverse of an m x n diagonal matrix 9. For a nonsymmetric matrix A= T- 1 AT, with A a diagonal matrix, it is
D with ~i = t1i6i; is the n x m diagonal matrix D' with ~i = ; 1 6,; not true in general that A' = T- 1 A'T is the pseudo-inverse. Find a 2 x 2
whenever q' :/: 0 and d~i = 0 otherwise. example which illustrates geometrically what goes wrong.
56 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES PROBLEMS FOR CHAPTER 1 57

10. Find the singular value decomposition and pseudo-inverse of Problem Section 1.6
1. A Write a computer program (in MATLAB) to find the largest eigen-
2v'S -2v'5) value of a matrix using the power method. Use this program to solve the
A= 3 3 .
( 6 6
following problems:
(a) Find the largest eigenvalue of the matrix in Problem 1.3.1.
11. Find the least squares solution of then linear equations (b) Find the smallest eigenvalue of the matrix in Problem 1.3.1. (Shift
the eigenvalues by subtracting an appropriate constant from the di-
a;x + b;y = Ci = 1,2, ... ,n,
i agonal.)
where a;b1 - a1b, :f: 0 for i :f: j. H r1, j = 1, 2, ... , k = n(n- 1)/2 are 2. ~ Develop a ranking scheme for your favorite sport. First collect the
the solutions of all possible pairs of such equations, show that the least data for all the games played between teams (or individuals).
squares solution
(a) Take a;i = 1 if team i beat team j and zero otherwise. Does this
r = (:) give a reasonable ranking?

is a convex linear combination of ri, specifically (b) Take aii to be the percentage of points earned by team i in the contest
between team i and j. Does this give a reasonable ranking?
k (c) Propose your own method to assign values to the matrix A.
r = LPJTj,
J=l 3. Prove that a diagonally dominant matrix has a nonnegative eigenvector.

where
D2
P - i
i - """
L..ti=l
D2'
i

and Di is the determinant of the jth 2-equation subsystem. Interpret this

result geometrically.

12. The matrices M~c, k = 1, 2, ... , n, represent elementary row operations if

they have the form

fflij =1 if i = j
m,k = -aile if i > k
m;1 =0 otherwise.

Suppose L is a triangular matrix with entries l;j = 1 if i = j, l,i = a,i if

=
i > j, l;j 0 otherwise. Show that (MnMn-1 · · · Mt)- 1 L. =
13. .1!. Find the SVD of then x n Hilbert segment A= (at1),au = ,~.,for
3
several values of n. Why is this matrix ill-conditioned?

14. (a) Show that IIAIIF = Tr(A* A) is a norm on the space of n x k matrices.
(b) Show that if A and B are related through an orthogonal similarity
transformation, then IIAIIF = IIBIIF·
(c) Show that IIAIIF = IIA*IIF, even though A and A* may have different
dimensions.
I
·~

i
:iJ
~
•'ffi

t
•
-~

I
:}§
-~

J Bibliography
j
·~
1t
~ [1] M. J. Ablowitz, P. A. Clarkson, and M. A. Ablowitz. Solitons, Nonlin-

I
ear Evolution Equations and Inverse Scattering, volume 149 of London
Mathematical Society Lecture Notes. Cambridge University Press, 1991.
{432}

·~::1
~ [2] M. J. Ablowitz and A. S. Fokas. Complex Variables: Introduction and
Applications. Number 16 in Cambridge Texts in Applied Mathematics.
.~
t
};~ Cambridge University Press, Cambridge, 1997. {273}
.~
~ [3] M. J. Ablowitz and H. Segur. Solitons and the Inverse Scattering Thins-
~
•';!
,
'i form. SIAM, Philadelphia, 1981. {432}
-_'4
.~ [4] M. Abramowitz and I. A. Stegun. Handbook of Mathematical Functions.
I Dover, New York, 1974. {402}
I [5] K. Aki and P. G. Richards. Quantitative Seismology, Theory and Methods:
I1" Vol. JJ. Freeman, 1980. {132}
'$
[6) R. Askey. Orthogonal Polynomials and Special Functions. SIAM, Philadel-
!
'$1
phia, 1975. {93}

[7] K. E. Atkinson. A Survey of Numerical Methods for the Solution of Fred-

holm Integral Equations of the Second Kind. SIAM, Philadelphia, 1976.
{127}

[8] G. K. Batchelor. An Introduction to Fluid Dynamics. Cambridge Univer-

sity Press, Cambridge, 1967. {488, 497}

[9] M.S. Bauer and D. S. Chapman. Thermal regime at the Upper Stillwater
dam site, Uinta Mountains Utah. Tectonophysics, 128:1-20, 1986. {408}

[10] C. M. Bender and S. A. Orszag. Advanced Mathematical Methods for

Scientists and Engineers. International Series in Pure and Applied Math-
ematics. McGraw-Hill, 1978. {463}

[11] P. G. Bergmann. Introduction to the Theory of Relativity. Dover, New

York, 1976. {203}

559
560 BIBLIOGRAPHY

[12] N. Bleistein and R. A. Handelsman. Asymptotic Expansions of Integrah.

Dover, New York, 1986. {463}
'1
~
·"~
BIBLIOGRAPHY 561

[29] I. Daubechies. Ten Lectures on Wavelets. SIAM, Philadelphia, 1992. {93,

328}
[13] A. Brandt. Multi-level adaptive solutions to boundary-value problems.
Math. Comp., 31:333-390, 1977. {402}

[14] W. L. Briggs. Multigrid Thtorial. SIAM, Philadelphia, 1987. {402}

I1 [30] C. deBoor. A Pmctical Guide to Splines, volume 27 of Applied Mathemat-
ical Sciences. Springer-Verlag, New York, 1978. {93}

[31] W. Eckhaus and A. Van Harten. The Inverse Scattering Transformation

and the Theory of Solitons. North Holland, Amsterdam, 1981. {432}
[15] J. Buckmaster and G. S. S. Ludford. Theory of Laminar Flames. Cam- il
?l
bridge University Press, New York, 1982. {551} i [32] A. Erdelyi. Asymptotic Expansions. Dover, New York, 1987. {463}
·'
·~
[16] G. F. Carrier, M. Krook, and C. E. Pearson. Functions of a Complex ' [33) P. C. Fife. Models for phase separation and their mathematics. In
Variable. McGraw-Hill, New York, 1966. {273} ~~ M. Mimura and T. Nishiuda, editors, Nonlinear Partial Differential Equa-
j tions and Applications. Kinokuniya Publishers, 1991. {402}
[17] K. M. Case. On discrete inverse scattering problems. J. Math. Phys., ll
·J
14:916-920, 1973. {431, 432}
1 [34] H. Flaschka. On the Toda lattice, II-inverse scattering solution. Prog.
~ . Theor. Phys., 51:703-716, 1974. {431, 432}
[18] B. W. Char, K. 0. Geddes, G. H. Gonnet, B. L. Leong, M. B. Monagan, 3
~1
and S.M. Watt. Maple V Language Reference Manual. Springer-Verlag, <<

~j
[35] G. E. Forsythe, M.A. Malcolm, and C. B. Moler. Computer Methods for
1991. {48}
~~
Mathematical Computations. Prentice-Hall, 1977. {48}
;j
[19] A. Cherkaev. Variational Methods for Structuml Optimization. Springer,
New York, 1999. {203} ~ [36] A. C. Fowler: Mathematical Models in the Applied Sciences. Cambridge
:l Texts in Applied Mathematics. Cambridge University Press, Cambridge,
1997. {551}
[20) Y. Choquet-Bruhat and C. DeWitt-Moretti. Analysis, Manifolds and
Phusics. North-Holland, Amsterdam, 1982. {497}
[21] S. N. Chow and J. K. Hale. Methods of Bifurcation Theory. Springer-
j [37) B. Friedman. Principles and Techniques of Applied Mathematics. Dover,
New York, 1990. {171, 328, 401}
Verlag, New York, 1982. {496} .; [38] P. R. Garabedian. Partial Differential-Equations. Chelsea Publishing Co.,
~
,')
[22] S. N. Chow, J. K. Hale, and J. Mallet-Paret. An example of bifurcation New York, 1998. {401}
to homoclinic orbits. J. Diff. Eqns., 37:351-373, 1980. {497}
I
~;j (39) I. M. Gel'fand and S. V. Fomin. Calculus of Variations. Prentice-Hall,
[23] C. K. Chui. An Introduction to Wavelets. Academic Press, 1992. {93, ~ Englewood Cliffs, 1963. {202}
~
307, 328}

[24) E. T. Copson. Theory of Functions of a Complex Variable. Oxford at the

l
'P
[40) I. M. Gel'fand and B. M. Levitan. On the determination of a differential
equation from its spectral function. Am. Math. Soc. Transl., pages 259-
Clarendon Press, 1935. {273}

[25) E. T. Copson. Asymptotic Expansions. Cambridge University Press, Cam-

1
'j
J
309, 1955. Series 2. {432}

[41] I. M. Gel'fand and G. E. Shilov. Genemlized Functions, volume 1. Aca-

bridge, 1967. {463} -~ demic Press, New York, 1972. {171}
·•
[26) A. M. Cormack. Representation of a function by its line integrals, with
some radiological applications. J. Appl. Phys., 34:2722-2727, 1963. {128}
~1 [42) H. Goldstein. Cl£Usical Mechanics. Addison-Wesley, Reading, MA, 2nd
edition, 1980. {203}
!
oW
[27] R. Courant and D. Hilbert. Methods of Mathematical Physics, volume 1. [43] G. H. Golub and C. F. Van Loan. Matrix Computations. Johns Hopkins
Wiley-Interscience, New York, 1953. {47, 171} ~'1 University Press, 3rd edition, 1996. {48}

[28] S. J. Cox and M. L. Overton. On the optimal design of columns against [44) D. Gottlieb and S. A. Orszag. Numerical Analysis of Spectral Methods;
buckling. SIAM J. Appl. Math., .23:287-325, 1992. {203} Theory and Applications. SIAM, Philadelphia, 1977. {401}
562 BIBLIOGRAPHY
(45] J. Guckenheimer and P. Holmes. Nonlinear Oscillations, Dynamical Sys-
tems, and Bifurcations of Vector Fields, volume 42 of Applied Mathemat-
ical Sciences. Springer-Verlag, New York, revised edition, 1990. {496}
(46] R. Haberman. Elementary Partial Differential Equations. Prentice-Hall,
3rd edition, 1997. {401}
(47] J. K. Hale. Ordinary Differential Equations. Wiley-Interscience, New
York, 1969. {513, 551}
(48] P.R. Halmos. Finite Dimensional Vector Spaces. Van Nostrand Reinhold,
New York, 1958. {47}
(49] D. Hanselman and B. Littlefield. Mastering Matlab. Prentice-Hall, Upper
Saddle River, NJ, 1996. {48}
[50] A. Heck. Introduction to Maple. Springer-Verlag, New York, 2nd edition,
1996. {48}
[51] P. Henrici. Applied and Computational Complex Analysis. Wiley and
Sons, 1986. {273}
[52] H. Hochstadt. Integral Equations. Wiley-Interscience, New York, 1973.
{127}
[53] H. Hochstadt. The FUnctions of Mathematical Physics. Dover, New York,
1986. {93}
[54] M. H. Holmes. Introduction to Perturbation Methods. Springer-Verlag,
New York, 1995. {551}
(55] F. Hoppensteadt. Mathematical Theories of Populations: Demographics,
Genetics and Epidemics. SIAM, Philadelphia, 1975. {127}
[56] R. A. Hom and C. R. Johnson. Matrix Analysis. Cambridge University
Press, Cambridge, 1985. {44, 47}
[57] P. Horowitz and W. Hill. The Art of Electronics. Cambridge Press, Cam-
bridge, MA, 2nd edition, 1989. {497}
[58] C. M. Hutchins. The acoustics of violin plates. Scientific American,
245:17Q-187, 1981. {401}
[59] F. M. Phelps Ill, F. M. Phelps IV, B. Zorn, and J. Gormley. An experi-
mental study of the brachistochrome. Eur. J. Phys., 3:1-4, 1984. {204}
(60] G. Iooss and D. Joseph. Elementary Stability and Bifurcation Theory.
Springer-Verlag, New York, 1989. {496}
(61] E. Jahnke and F. Emde. Tables of FUnctions. Dover, New York, 1945.
{402}
BIBLIOGRAPHY 563
[62] 0. Kappelmeyer and R. Haenel. Geothermics with Special Reference to Ap-
plication. Number No 4 in Geoexploration Monographer, Ser. 1. Gebriider
Borntraeger, Berlin, 1974. {401}
[63] J. Keener and J. Sneyd. Mathematical Physiology. Springer-Verlag, New
York, 1998. {551}
(64] J.P. Keener. Analog circuitry for the van der Pol and Fitzhugh-Nagumo
equations. IEEE 7rans on Sys., Man, Cyber., pages 101Q-1014, 1983.
SMG-13. {497}
(65] J. P. Keener. The Perron Frobenius theorem and the ranking of football
teams. SIAM Rev., 35:8Q-93, 1993. {48}
[66] J. B. Keller. Inverse problems. Am. Math. Monthly, 83:107-118, 1976.
{432}
[67] J. B. Keller and S. Antman. Bifurcation Theory and Nonlinear Eigenvalue
Problems. Benjamin, New York, 1969. {496}
[68] J. Kevorkian and J. D. Cole. Multiple Scale and Singular Perturbation
Methods, volume 114 of Applied Mathematical Sciences. Springer-Verlag,
Berlin, 1996. {546, 551}
(69] S. Kida. A vortex filament moving without change of form. J. Fluid Mech.,
112:397-409, 1981. {497} .
(70] D. J. Korteweg and G. deVries. On the change of form of long waves
advancing in a rectangular canal, and on a new type of long stationary
waves. Phil. Mag., 39:422-443, 1895. {432}
[71] Y. Kuznetsov. Elements of Applied Bifurcation Theory. Springer-Verlag,
New York, 2nd edition, 1998. {496}
(72] G. L. Lamb, Jr. Elements of Soliton Theory. John Wiley and Sons, New
York, 1980. {328, 432, 497}
[73] L. D. Landau and E. M. Lifshitz. Mechanics. Butterworth-Heineman, 3rd
edition, 1976. {203}
[74] C. L. Lawson and R. J. Hanson. Solving Least Squares Problems. SIAM,
Philadelphia, 1995. {48}
[75] P. Lax. Integrals of nonlinear equations of evolution and solitary waves.
Comm. Pure Appl. Math., 21:407-490, 1968. {432}
(76] G. A. Lion. A simple proof of the Dirichlet-Jordan convergence theorem.
Am. Math. Monthly, 93:281-282, 1986. {165}
(77] W. V. Lovitt. Linear Integral Equations. Dover, New York, 1950. {127}
564 BIBLIOGRAPHY 565
BIBLIOGRAPHY

[78] J. Lund and K. L. Bowers. Sine Methods for Quadrature and Differential
Equatiom. SIAM, Philadelphia, PA, 1992. {93, 274}
[79] W. S. Massey. Algebraic Topology: An Introduction. Springer-Verlag, New
York, 1990. {497}
[80] S. F. McCormick, editor. Multigrid Methods. Frontiers in Applied Math-
ematics. SIAM, Philadelphia, 1987. {402}
[94] T. D. Rossing. The physics of kettledrums. Scientific American, 247:172-
179, Nov. 1982. {401}
[95] W. Rudin. Principles of Mathematical Analysis. McGraw-Hill, New York,
3rd edition, 1976. {92, 165}
[96] W. Rudin. Real and Complex Analysis. McGraw-Hill, New York, 3rd
edition, 1987. {164}

[81] L. M. Milne-Thomson. Theoretical AerodJmamics. Dover, New York, 1973.
{274}
[82) J. D. Murray. Asymptotic Analysis. Springer-Verlag, New York, 1984.
{463}
[83) J. D. Murray. Mathematical Biology. Springer-Verlag, New York, 1989.
{402}
[84) F. Natterer. The Mathematics of Computerized Tomography. Wiley, New
York, 1986. {128}
[85] A. W. Naylor and G. R. Sell. Linear Operator Theory in Engineering and
Science. Holt, Rinehart and Winston, New York, 1971. {127} ·
[86) F. W. J. Olver. Asymptotics and Special F'u.nctions. A. K. Peters, Ltd.,
London, 1997. {463}
[87] R. E. O'Malley, Jr. Introduction to Singular Perturbations. Academic
Press, New York, 1974. {551}
[88) M. Pickering. An Introduction to Fast Fourier Transform Methods for
Partial Differential Equations, with Applicatiom. Wiley and Sons, New
York, 1986. {401}
[97] D. R. Smith. Singular Perturbation Theory. Cambridge University Press,
Cambridge, 1985. {551}
[98] I. N. Sneddon. Fourier Transforms. Dover, New York, 1995. {328}
[99] I. Stakgold. Branching of solutions of nonlinear equations. SIAM Rev.,
13:289-289, 1971. {496}
[100] I. Stakgold. Boundary Value Problems of Mathematical Physics. Macmil-
lan, New York, 2nd edition, 1998. {171, 328}
[101] F. Stenger. Numerical methods based on Whittaker cardinal, or sine
functions. SIAM Rev., 23:165-224, 1981. {274}
[102] F. Stenger. Numerical Methods Based on Sine and Analytic F'u.nctions,
volume 20 of Springer Series in Computational Mathematics. Springer-
Verlag, New York, 1993. {93, 274}
[103) G. W. Stewart. Introduction to Matrix Computations. Academic Press,
New York, 1980. {48}
[104] J. J. Stoker. Nonlinear Elasticity. Gordon and Breach, New York, 1968.

[89] D; L. Pqwers. Boundary Value Probletm. Academic Press, 4th edition,

l {203}

[105) J. J. Stoker. Differential Geometry. Wiley-Interscience, New York, 1969.

{202, 497}
1999. {401}

(90] W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling.

Numerical Recipes in Fortran 77: The Art of Computing. Cambridge
l.,
'.:ti
[106] G. Strang. Linear Algebra and its Applications. Academic Press, New
York, 3rd edition, 1988. {47}

University Press, Cambridge, 1992. {48, 77, 402} [107) G. Strang and G. Fix. An Analysis of the Finite Element Method. Prentice-

[91] R. H. Rand. Topics in Nonlinear Dynamics with Computer Algebra. Gor-

l
~
~
Hall, 1973. {93}

don and Breach, Langhorne, PA, 1994. {497} [108] G. Strang and T. Nguyen. Wavelets and Filter Banks. Wellesley-
~~
1 Cambridge Press, Wellesley, MA, 1997. {93, 328}
[92] R. H. Rand and D. Armbruster. Perturbation Methods, Bifurcation The-
ory and Computer Algebra, Applied Mathematical Sciencu, volume 65. ~
~~
(109] A. F. Timin. Theory of Approximation of Functions of a Real Variable.
Springer-Verlag, New York, 1987. {497} :1 Macmillan, New York, 1963. {328}
:1J'l
(93] G. F. Roach. Green's FUnctions: Introductory Theory with Applications. [110) E. C. Titchmarsh. Theory of F'u.nctions. Oxford University Press, 1939.
Van Nostrand Reinhold, London, 1970. {171} J~ {273}
566 BIBLIOGRAPHY

[111] E. C. Titchmarsh. Eigenfunction Expansions Associated with Second-

Order Differential Equations. Oxford University Press, Oxford, 1946.
{328}
[112] L. N. Trefethen and D. Bau. Numerical Linear Algebra. SIAM, Philadel-
phia, 1997. {48}
[113] F. G. 'Iiicomi. Integral Equations. John Wiley and Sons, New York, 1957. Appendix A
{127}
(114) A. M. Turing. The chemical basis of morphogenesis. Phil. 7rans. Roy.
Soc. Lond., B237:37-72, 1952. {390}
Selected Hints and
[115] M. Van Dyke. Perturbation Methods in Fluid DynamiC8. Academic Press,
New York, 1964. {551} Solutions
[116) M. Van Dyke. An Album of Fluid Motion. The Parabolic Press, Stanford,
1982. {274}
j
(117) R. Wait and A. R. Mitchell. Finite Element Analysis and Applications.
Wiley and Sons, New York, 1968. {93} [jt.l.l; Hint: What if one basis was smaller than another?
[118] J. W. Ward and R. V. Churchill. Fourier Series and Boundary Value i 1.1.3; (a) Follows from direct verification.
Problems. McGraw Hill, New York, 5th edition, 1993. {92} , (b) Follows from (a). H the norm is known to be induced by an inner
'! product, then (a) shows how to uniquely calculate the inner product.
(119] H. F. Weinberger. Variational methods for eigenvalue approximation.
SIAM, 1974. {203} (c) Suppose llxll = c~=~llxkiP)l/p.
(-¢:::) If p = 2, then (x, y) = l':~=l XkYk induces the norm.
[120] H. F. Weinberger. A First Course in Partial Differential Equations. Dover, (:::}) If the norm is induced by an inner product, then from (a)
New York, 1995. {401}
(121} G. B. Whitham. Linear and Nonlinear Waves. Wiley and Sons, New
York, 1974. {401}
(x, y) = 41 ( (L
n
ixk + YkiP) 21P- (L ixk- YkiP) 21P
. n )
.
k-1 k=l
[122) J. H. Wilkinson. The Algebraic Eigenvalue Problem. Oxford University Take x = (1, 0, 0, ... , 0), and y = 0, 1, 0, ... , 0). Then (x, x) =
Press, London, 1965. {47} 1, (x, y) = 0, and (x, x + y) = !( (2P + 1?1P - 1). Since for an inner
[123] F. A. Williams. Combustion Theory. Addison-Wesley, Reading, Mass., product, (x, x + y) = (x, x) + (x, y), it must be that (2P + 1) 2 /P = 5.
1965. {551} Since (2P + 1) 2 /P is a monotone decreasing function of p which ap-
proaches 1 for large p and is unbounded at the origin, the solution of
[124) T. J. Willmore. An Introduction to Differential Geometry. Oxford Uni- (2P + 1) 2 /P = 5 at p = 2 is unique. We conclude that p = 2.
versity Press, London, 1959. {203}
1.1.4; (F,F) = 0 does not imply that f(x) = 0.
[125) S. Wolfram. Mathematica. Addison-Wesley, Reading, MA, 3rd edition,
1996. {48} 1.1.8; Observe that with {3 = (x, y) /llvll 2 , x- {3y is orthogonal toy, so that
(126] E. Zauderer. Partial Differential Equations of Applied Mathematics. Wiley ilx- avW = llx- f3vW + li(a- f3)vW,
and Sons, New York, 2nd edition, 1998. {401}
which is minimized when a= {3. Clearly, if x = ay, then
[127) A. H. Zemanian. Distribution Theory and 7ransform Analysis: An Intro-
duction to Generalized Functions, with Applications. Dover, New York, i(x,v)l 2 = llxWIIvW·
1987. {171}
If so, then we calculate directly that llx- {3yjj 2 = 0, so that x = {3y.

567
566 BIDLIOGRAPHY

[111] E. C. Titchmarsh. Eigenfunction Expansions Associated with Second-

Order Differential Equations. Oxford University Press, Oxford, 1946.
{328}
[112] L. N. Trefethen and D. Bau. Numerical Linear Algebra. SIAM, Philadel-
phia, 1997. {48}
[113) F. G. 'llicomi. Integral Equations. John Wiley and Sons, New York, 1957. Appendix A
{127}
[114] A. M. Turing. The chemical basis of morphogenesis. Phil. Thlns. Ro1J.
Soc. Lond., B237:37-72, 1952. {390} Selected Hints and
[115] M. Van Dyke. Perturbation Methods in Fluid Dynamics. Academic Press,
New York, 1964. {551} Solutions
[116] M. Van Dyke. An Album of Fluid Motion. The Parabolic Press, Stanford,
1982. {274}
[117) R. Wait and A. R. Mitchell. Finite Element Analysis and Applications.
1.1.1; Hint: What if one basis was smaller than another?
Wiley and Sons, New York, 1968. {93}
[118] J. W. Ward and R. V. Churchill. Fourier Series and Boundary Value 1.1.3; (a) Follows from direct verification.
Problema. McGraw Hill, New York, 5th edition, 1993. {92} (b) Follows from (a). If the norm is known to be induced by an inner
product, then (a) shows how to uniquely calculate the inner product.
[119] H. F. Weinberger. Variational methods for eigenvalue approximation.
SIAM, 1974. {203} (c) Suppose llxll = o=~=llxkiP) 1 1P.
(-<=) If p = 2, then (x, y} = L:~=l XkYk induces the norm.
(120] H. F. Weinberger. A Firat Course in Partial Differential Equations. Dover, (:::::?)If the norm is induced by an inner product, then from (a)
New York, 1995. {401}
(121] G. B. Whitham. Linear and Nonlinear Waves. Wiley and Sons, New
York, 1974. {401}
1(
(x, Y} = 4 (L lxk + YkiP) 21P- (L lxk- YkiP) 21P
n n )
.
k-1 k=l
[122] J. H. Wilkinson. The Algebraic Eigenvalue Problem. Oxford University Take x = (1, 0, 0, ... , 0), and y = 0, 1, 0, ... , 0). Then (x, x} =
Press, London, 1965. {47} 1, (x, y} = 0, and (x, x + y} = ~ ( (2P + 1)2/P - 1). Since for an inner
[123] F. A. Williams. Combustion Theory. Addison-Wesley, Reading, Mass., product, (x, x + y} = (x, x} + (x, y), it must be that (2P + 1)2 /P = 5.
1965. {551} Since (2P + 1)2/P is a monotone decreasing function of p which ap-
proaches 1 for large p and is unbounded at the origin, the solution of
[124) T. J. Willmore. An Introduction to Differential Geometry. Oxford Uni- (2P + 1) 2 /P = 5 at p = 2 is unique. We conclude that p = 2.
versity Press, London, 1959. {203}
1.1.4; (F, F} = 0 does not imply that f(x) = 0.
[125) S. Wolfram. Mathematica. Addison-Wesley, Reading, MA, 3rd edition,
1996. {48} 1.1.8; Observe that with {3 = (x, y) /llvll 2 , x- {3y is orthogonal toy, so that

[126) E. Zauderer. Partial Differential Equations of Applied Mathematics. Wiley llx- ayll 2 = llx- f3Yll 2 + ll(o:- f3)yjj 2 ,
and Sons, New York, 2nd edition, 1998. {401}
which is minimized when a= {3. Clearly, if x = ay, then
[127] A. H. Zemanian. Distribution Theory and Thlnsform Anal1Jais: An Intro- 2
duction to Generalized FUnctions, with Applications. Dover, New York, l(x,y)j 2 = llxii IIYW·
1987. {171}
If so, then we calculate directly that llx- {3yjj 2 = 0, so that x = {3y.

567
 568 APPENDIX A. SELECTED HINTS AND SOLUTIONS 569

1.1.9; (a) ¢o(x) = 1, c/J1(x) = x, c/J2(x) = x 2 - ~, ¢a(x) = x3 - ~x, ¢4(x) = (d) T does not exist.
/x4- li.x2
7
+ .1...
35.
1 1
(b) ¢o(x) = 1,¢1(x) = x,¢2(x) = x 2 - ~ = ~cos(2cos- 1 x),¢3 (x) = 1
(e) T = -1 1 } ) , T- 1 AT = ( 0
0 0
~ 0
~).
x - ~x =-! cos(3cos- 1 x), ¢4(x) = x 4 - x 2 + k = kcos(4cos- 1 x).
3
( 0 -v'3 ~ 0 0 3
(c) ¢o(x) = 1,¢1(x) = x -1,¢2(x) = x 2 - 4x + 2,¢3 (x) = x 3 - 9x 2 +
18x- 6, ¢4(x) = x 4 - 48x 3 + 72x 2 - 96x + 24.
1.2.7; T= ( _1 _2 ) ,T-1 AT= ( 3 0) .
O
(d) ¢o(x) = 1,¢1(x) = x,¢2(x) = x 2 - !,¢a(x) = x 3 - ~x,¢4(x) = 2 1 6
x4- 3x2 + i·
1.2.8; If x is in M, then Px = x, and if x is in the orthogonal complement of
1.1.10; ¢o(x)
33 2
= 1,¢1(x) = x,¢2(x) = x - ~,¢a(x)
27 ... ( ) 5 1930 3 4411
2
= x3 - t x,¢4(x) =
0
x4 - M, the Px = 0. Therefore, P has two eigenvalues, A= 0, 1.
28 X + 140' '1'5 X = X - 'i359X + 1057X.
~ ~

n
1.2.10; (d) Examine the eigenvectors of A and A* where A= ( ).
2 3
1.2.1; (a) Relative to the new basis, A= 1 1
(
0 0 1.3.1; Hint: Minimize (Ax,x) with a vector of the form xT = (1, -1,z,O).
(b) Set C = 11 21 43 ) , and D = ( 01 1 0 0) 1 . Then the repre-
1.3.2; (a) Prove that if the diagonal elements of a symmetric matrix are in-
creased, then the eigenvalues are increased (or, not decreased) as
( 2 3 1 0 -1 1 well.
sentation of A in the new basis is
4 4 )
(b) Find the eigenvalues and eigenvecto"' of B = ( : 8 -4 , and
4 ) 1"
A'= v-'cAc-'v = (
53

~ ~'~
19

=! . ¥,

i use them to estimate the eigenvalues of A.

-4 8

1 23)
1.2.2; (c) A = ( 01o)
1 , and B
are not equivalent.
(lo)
0 2 have the same determinant but
=
l 1.3.3; The matrix

Apply 1.3.2a.
(3 2 2 4
4 7
has a positive, zero, and negative eigenvalue.

1.2.3; (a) Notice that if ABx =Ax, then BA(Bx)

(b) If AA•x =AX, then A(x, x) = (AA*x, x) = (A*x, A*x) :2: 0.
= A(Bx). ·~ 1.4.1; (a) b must be orthogonal to (1, 1, -1)T, and the solution, if it exists, is
unique.
1.2.4; If Ax= AX then A(x, x) = (Ax, x) = (x, AT x) = -(x, Ax) = -X(x, x}. (b) The matrix A is invertible, so the solution exists and is unique.

1.2.5; (a) R(A) = {(1,1,2)T,(2,3,5)T},N(A) = O,R(A*) = m?,N(A*) = 1.4.2; b must be in the range of P, namely M.
{(1, 1, -1)T}.
1.4.3; ( =>) Suppose A is invertible, and try to solve the equation ~i ai</Ji =

cn
0.
(b) R(A) = R(A*) = IR 3 , N(A) = N(A*) = 0. Taking the inner product with <Pi, we find 0 = Aa, so that a = 0, since

1.2.6; (a) T = ( i 1~ 01 ) , T- 1AT =

1 0
0 01
0 0
the null space of A is zero.
(¢:) Suppose {<Pi} form a linearly independent set and that Ax = 0. Then
(x, Ax} = (~i Xi</Ji, ~i xi <Pi) = 0, so that ~i Xi</Ji = 0, implying that
x = 0, so that A is invertible (by the Fredholm alternative).
(b) T = ( ~ ~i) ,T- 1 AT= ( ~i ~).
1.4.4; Since (Ax, x) = (x, A*x) > 0 for all x =J:. 0, the null spaces of A and A •

(c) T= ( ~1 -10
1 00) , T-1AT=
1 .
coo) 0 2 0
0 0 3
.
must be empty. Hence, (b, x) = 0 for all x in N(A*) so that Ax = b has
a solution. Similarly, the solution is unique since the null space of A is
empty.
 570 APPENDIX A. SELECTED HINTS AND SOLUTIONS 571

1.5.1; (a) A'= i ( 2 -1

-1
1) 2 1
2 •. - ~n
1 3 ., F1or Xn- 1 th .l!Ll' 1 1 _ ~m 1

~ (n~t)l 1 ~'! < (n!t)l is arbitrarily small for m and n large.

1 ~m-1 1
~1.:= 1 kf• e wuerence Xn-Xm - ~k=n+t kf ~ (n+ 1 )1 ~k=O ~

0 0 0

(b) A'= 2~ ( i ! ~) 2.1.4; The functions

independent.
{sinmrx}~= 1 are mutually orthogonal and hence linearly

(c) A' = 1\ ( :
0
~
-4
)
2.1.5; maxt lfn(t) - fm(t)l
form and n large. However,
= !(I- '!!;) if m > n, which is not uniformly small
J; lfn(t)- fm(t)!Zdt = (~2:':~ 2

< 1 ~n.

(d) A' = i ( -6 2 2)
-5 4 1
2.1.6; (a) Hint: Show that l(lul-lvl)l ~ ju- vi.
-4 2 2 2.1.8; J; X(t) = 0.
1.5.3; (a) Take u = :c -v.
2.1.11; J; (I; f(:c,y)dx) dy = - J; (I; f(:c,y)dy) dx = -~. Fubini's theorem

1.6.4; Q = (~,.P,,.P,), where~ = ;ky (:) ,¢>. = }.;. ( ~0 ) ,.P, = fails to apply because J; (I; lf(x, y)ldx) dy = f; (I; a:'!v' dx) dy =
J;; tan- ;dy does not exist.
-(v'ffi *
1

1(
75
2
~1 ) , and R- ¥) .
1Ji5 1f
~ 2.2.1; With w(x) = l,p(z) = \:x 2 + 136 , with w(:c) = v'f='?,p(x)
1), and with w(x) = ~,p(:c) = 3~(4x2 + 1).
= 1:1T (6x2 +
1.5.5; Use that
2.2.2; Define the relative error e~,: by e~ =1- ~· where /A:(x) is the kth
1 ( 1 -2 ) ( ViO 0 ) ( 1 -1 ) partial sum, f~.:(x) =~!::a ant/>n(x). Then
A = ViQ 2 1 0 2v'iQ . 1 1
or (a) H(:c) - 2H(x -?r) = ~ ~::0 2n~l sin(2n + l)x, e5 = 0.201,
A= ( i ~ ) ( ~ ~ ) ( ~ !a ) · (b) :c -?r = -2~~ 1 ~ sinnx, e5 = 0.332,
< X < 11" 4 ~oo
(c) { 211" X- x 11"0 < x < 2?r = 2 - 1T ~n=o
1r 1 (
cos 2n +
1) -
1.5.6; Use that A = i ( ~ ~2 ) ( ~ ~ ) ( !2 ~ ),so that (2n+t)!l x, e5 -
0.006,
AI = 1( -2} -2)
5 4 . l (d) x2 (2?r- x) 2 = 81~· -48 ~:: 1 ~ cosnx, es = 0.002.
~

1.5.8; For A = ( ~:: ~::~ ) , singular values are ViO and eVlO with E =
1 . f:t'n I
0.00 v LV, A =
( 0.} +
0.1 -
Jw
:Jrn =
0.2 -
0.2 +
~
:Jrn ) ( 200.1
-199.8
-99.8 )
100.2 .
'
j

~'
D.J

0.0

:1 .... ·1

Using instead singular values ViO and 0, A' = ( ~:~ ~:; ) .

-~

l
~
·1.0

0.0 D.J 1.0

:til
1.1 2.0 0.0 D.J !.0
no:
,.. 2.0

1.5.10; A= UJJV where U = ~ v'5 o -2

0 -1 o) , I:= ( 2V10
0 o ),
3VW j
Figure A.1: Left: Partial sum f~.:(x) for Problem 2.2.2a with k = 5; llight:
( 0 -2 1 0 0 Partial sum f~.:(x) for Problem 2.2.2b with k = 5.
1 ..Yi
V=~
1
( _ 1 -1
_ 1 ) , so that A I = ( _rl
20 SO
1 U1
.!. )
.
. 20
1
30 15 2.2.3; f(x) =x(x-?r)(x-211") = 12~~ 1 ;?s-sinnx.
 573
572 APPENDIX A. SELECTED HINTS AND SOLUTIONS

:u 100 2.2.15; Suppose the transforms of j,g, and hare fJ_, and respectively. Use F•
:u 10 direct substitution and the fact that
1
-k
L,i=~ e 21riJk/N = 1 if k is an
1.0 10 integer multiple of N (including 0), and = 0 otherwise to show that hj =
u
40
1.0 h9i·
10
o.a 2.2.16; Rewrite the definition of the discrete Fourier transform as a matrix mul-
0.0
0.0
. o.a 1.0
.til
1.1 1.0 CLO CL6 1.0
.til
... 1.0
tiplication. Show that the matrix is orthogonal.
2.2.19; See Fig. A.4.
Figure A.2: Left: Partial sum fk(x) for Problem 2.2.2c with k = 5; llight:
Partial sum /k(x) for Problem 2.2.2d with k = 3. 0.4

0.2
2.2.4; With the additional assumption that the function f(x) to be fit intersects
o.o-1---~
the straight line at the two points x = x 1 and x = x2, x1 < x 2, we must
have that x2- Xt =!,and x2 + x1 = 1, so that a=£/(})- !f(~),p = .0.2

2/(f)- 2f(i>· .(),4

2.2.6; (a) You should have a procedure that generates the Legendre polynomi- -2 ·1 0 3
X
als from Problem 1.1.9, or use Maple, which knows about Legendre
polynomials.
Figure A.4: The wavelet generated by the B-spline N 3 (x), for Problem 2.2.19.
I.Q I.Q

o.a ... 2.2.24; Solve the equation Aa = P where

0.0

....
0.0

.... A=
(
21 0)
1 4 1 ,
/o-
P= -3 ( /o -h
h) ·
0 1 2 h-h
.... 0.0 CL6 1.0
•1.0
·1.0 .... 0.0 CL6 1.0
X X
The solution is
Figure A.3: Left: Legendre polynomials P 1 (x),P2(x) and P3 (x); llight: Legen-
dre polynomials P4 (x),P5 (x).
a=
(
_2
_112 !0 -~1! ) ( h lo) .
i -! i h
(b) g(x) = ax+bx3 +cx5 , where a= ~(1r4 -15311"2 + 1485) = 3.10346, 2.2.26; (a) Solve Aa = P where
b= -m<11"
4 2
- 12511" + 1155) = -4.814388, c = ~(11" _ 10511" +
4 2
1¢~'1//jdx, 11 t/J~'t/ljdx.
945) = 1.7269. N

=
(c) g(x) 3.074024x-4.676347x 3 + 1.602323x5 • A plot of g(x) is barely
distinguishable from sin 'II"X on the interval -1 :$ x :$ 1.
aij =
1 0
Pi=- ~h
i=O 0

Use Problem 2.2.23 to evaluate the coefficients, and find that the ith
2.2. 7; Use integration by parts to show that the Fourier coefficients for the two equation is at-1 + 4at + ai+l = *(fi+l- /i-t), fori ::fi 0, N.
representations are exactly the same for any function which is sufficiently {b) Solve Aa = P where
smooth.
2.2.9; (b) Write tPn+l - AnXtPn = r:,:=o
PktPk, and evaluate the coefficients by
taking inner products with tPb and using part (a), and the fact that llt/Jkll =
aij = 1¢~¢jdx,
1

0
·
PJ = -
N
L:
~0
/i 1t/J~t/Jjdx.
0
1

1. Show that Bn = -An(XtPn 1 tPn) 1 On= An/An-1· Use Problem 2.2.23 to evaluate the coefficients, and find that the ith
2.2.14; Direct substitution and integration yields h(t) = 211" r:,:._ fkgkeikt. equation is -ai-1 + 8ai- at+l = lUi+l- /i-t), fori ::fi O,N.
00
 574 APPENDIX A. SELECTED HINTS AND SOLUTIONS 575

(c) Require ai-l+ 4ai + Gi+t = *Ui+t -/i-t), fori =f 0, N. 3.4.3; An = n281r2 is a double eigenvalue with ¢n(x) =sin n;x, '1/Jn(x) =cos n~x
(d) Solve Aa = f3 where for n odd.

= k(y,x):~~~·
Uii = 1'1/J~'(x)'l/J'j(x)dx,
1 3.4.4; k*(x,y)

3.5.1; (a) u(x) = f(x) +fox ex-tf(t)dt =ex when f(x) = 1.

(b) u(x) = f(x) + J; sin(t- x)f(t)dt = cosx when f(x) = 1.
/3i = 1
0
1
g(x)'l/Ji(x)dx-
N
2.:::: /i
i=O
1 0
1
</Ji(x)'l/lj(x)dx. (c) u(x) = f(x) + J; sin(x- t)f(t)dt =ex when f(x) =I-+ x.
3.5.2; (a) u(x) = f(x) + 1 ~>. + f01 f(t)dt = x + 2 ( 1~>.) when f(x) = x, provided
2.2.27; Observe that the equations for a 1, ... , GN-1 from Problem 2.2.26a and c
are identical.
A# 1.
+ ~ f 0 xtf(t)dt = x when f(x)
1
(b) u(x) = f(x) = 5: .
3.1.1; Use Leibniz rule to differentiate the expression u(x) =I 1
y(x-l)f(y)dy+
(c) u(x) = p(x) + a(x) I:o exp (-I: b(t)a(t)dt) b(y)p(y)dy.
I: x(y- l)f(y)dy twice with respect to x.
0

II>.Kr+
1

3.2.1; Find a sequence of functions whose £ 2 norm is uniformly bounded but 3.6.1; (a) Show that llu- unll < "["
1 _ j>.KII ·
whose value at zero is unbounded. There are plenty of examples. (b) Use that IIKII :::; ! to
conclude that n 2: 2. The exact solution is
2
u(x) = 3~~ and some iterates are u 0 (x) = 0, u1(x) = 1, u2(x) = {,
3.2.2; The proof is the same for all bounded linear operators; see page 107. ua(x) = 9 .
3.2.3; (a) The null space is spanned by u = 1 when A= 2, therefore solutions 3.6.2; If f(x"') = 0, for convergence to x*, require II- f'(x"')l < 1 or 0 < f'(x"') <
exist and are unique if A =f 2, and solutions exist (but are not unique) 2.
112
if A= 2 and I 0 f(t)dt = 0. 1f(x·v"<x·) 1 < 1, wh'ICh 1'f f( x "') = o, and
3 •6 . 3 ; ror
D
convergence to x • , reqmre•
1 fl , . \'1 2
(b) The null space is spanned by u = x when A= 3, therefore solutions
f'(x*) =f 0 is always satisfied.
exist and are unique if A =f 3, and solutions exist, but are not unique,
1
if A= 3 and J0 tf(t)dt = 0. 3.6.4; (a) Let Xn+l = D- 1 b- D- 1 Rxn-

*
(c) The null space is spanned by ¢(x) = cosjx if A =
if A =f
Therefore, *· 3.6.5; Show that Un(x) = 2:::~=0 ~ ·
* for j = 1, · · ·, n, the solution exists and is unique, while if
2
A = for some j, then a solution exists only if J0 '/r f (x) cos j xdx = 0. 3.7.1; y = y(x) where a 2x = sin- y'y + yJl- y2, a=~·
1

2 3.7.2; (a) A straight line.

3.3.1; u(x) = f(x)+A I 0 '/r 2:}= 1 i-~'lr cosjtcosjxf(t)dt = sin 2 x- ~ 2_1r>.1r cos 2x,
provided A =f :. For A=:, the least squares solution is u(x) = ~· (b) y = y(x) where x = hJY2=1-
~ ln(y+ Jy 2 -1), provided y 2: 1,
which means this problem is not physically meaningful, since y cannot
3.3.2; u(x) = 3 _!6 >.Po(x) + 3 ! 2>.P1(x) + 15 ~6 >.P2(x), provided A =f ~~~· It is reach zero.
helpful to observe that x 2 + x = !Po(x) + P1(x) + ~P2(x). 3.7.3; (a) T(p) = I;(p) c(z) V 1 ~dz.
1-c~(z)p

3.4.1; (a) Eigenfunctions are ¢n(x) = sinn1l'x for An= n 2~ 2 . (b) Let y = ljc 2(z), then use Abel's technique to show that z(c) =
(b) f(x) = -;!-r E~o (~~~\)2 sin(2n- 1)11'x. IJl/c ~
-:; 1/co yc2p2-t'

3.4.2; (b) </11 (x) = sinx, At = ~' ¢2(x) = cosx, A2 = ~'/r. (c) T(x) = ~lnJl + y 2 - y, where y = ~~·
(c) There are no eigenvalues or eigenfunctions. Remark: The existence tsin(k+~)n d
of eigenfunctions is only guaranteed for self-adjoint operators.
4.1.1; ( a) Use that Sk ( x) = 2 sin if for -1 < x < 1, an then observe that

(d) ¢n(x) = sinanx, An= ai•

where an= 2 1 . n:f siV¥ Sk(x) = (k + !) sinc((k + !)x) is a delta sequence.
 577
576 APPENDIX A. SELECTED illNTS AND SOLUTIONS

4.1.4; Observe that X= f~oo 1/l(x)dx is a test function, x(O) = 0, so that X= x</J
1i 4.2.12; g(x,y) = -!e-lv-xl, u(x) = ~ J.:O e-lv-xlu(y)dy- f~oo !e-lv-xlf(y)dy.
for some test function </J. Hence, 1/J(x) = Jx (x<P(x)). ' 4.2.13; u(x) = cosx +).. J; p(e) sin(x- e)u(e)~.
4.1.5; u(x) = c1 + c2H(x) + ca5(x). Show that a test function 1/J is of the form 1
1/J = .Jx(x2</J) if and only if f~oo 1/Jdx = f 0 1/Jdx = 1/J(O) = 0.
00 j
l
4.2.14; u(x) = 1 + J; eln(Vu<e)~.
4.1.6; u(x) = 5(x) + c1x + c2. ~ 4.3.1; L*v = v"- (a(x)v)' + b(x)v, with boundary conditions v(O) + v'(l)-
! a(l)v(l) = 0, v(l) + v'(O)- a(O)v(O) = 0.
4.1.7; Set u = xv, so that x 2v' = 0, and then u(x) = c1x + c2xH(x) (using that
x5(x) = 0). 4.3.2; L*v = -(p(x)v')' + q(x)v with p(O)v(O) = p(1)v(1), and p(O)v'(O) =
p(l)v' (1).
4.1.8; u(x) = c1 + c2H(x).
4.3.3; L*v = v"- 4v'- 3v with v'(O) = O,v'(1) = 0.
4.1.9; 5(x2 - a 2) = 2lal(5(x- a)+ 5(x +a)).
4.3.4; (a) g*(y,x)w(y) = (g*(e,x),<>(e- y))e = (g*(e,x),Len(e,v))e
4.1.10; In the sense of distribution, x'(x) = 5(x)- 5(x- 1), since (x'(x)<P(x)) = = (Leg*(e, x), g(e, y))e =(<>(e-x), g(e, y))e = g(x, y)w(x).
1
-(x(x), <P'(x)) =-I: x(x)<P'(x)dx =- f 0 <P'(x)dx = <P(o)- </1(1). (b) u(x)w(x) =(<>(e-x, u(e))e = (Leg*(e, x), u(e))e = (g*(e, x).J(e))e
4.1.11; (a) For distributions f and g, define (! * g, <P) = (g, 1/J), where 1/J(t) = = w(x) J: g(y, x)f(x)dx.
(f(x), </J(x + t)).
4.3.6; This follows from Problem 4.3.2.
{b) 5*5=5(x).
4.2.1; g(x,y) = -x for 0 ~ x ~ y,g(x, y) = g(y,x). 4.3.7; M*y =!fit+ AT(t)y, with the vector ( ~~~~~ ) E N..L([L, R]).

4.2.2; U(x) =xis a solution of the homogeneous problem. There is no Green's 4.3.8; (a) The operator is formally self-adjoint, but not self-adjoint.
function.
(b) Require u'(O) = u'(l) = 0, for example.
4.2.3; g (x, y ) = cosa(;-lx-v\)
111n f prov1'ded a .,...
..J..
2mr.
4.3.9; Require J:1r f(x) sinxdx =a, and J:1r f(x) cosxdx = {3.
for 0 $ x < y ~ 1
-(1- y)2x
4.2.4; g(x, Y) = { (2 _ y)yx- y for 0 ~ y < x ~ 1 1
~.3.10; Require f 0 f(x)dx = -{3.
4.2.5; g(x,y) = lny for x < y, g(y,x) = g(x,y). 4.3.11; Require J0! f(x) sin 1rxdx = {3 + 1ra.
_x(3y5/2+2) for 0 ~ x < y 4.3.12; Require Jg f(x)dx =a- {3.
4.2.6; g(x, y) ={ 3tf( -a/ )
- 11 5 3x+2x 2 for x > y
4.4.1; g(x,y) = yx+~(y-x)+l2 -~(x2 +y2 ),for0 ~ x < y ~ l,g(x,y) = g(y,x).
4.2.7; u(x) = -£:. f~oo e-alx-eiJ(e)cJe.
4.2.8; The functions sin 2x and cos 2x are elements of the null space of the op-
4.4.2; g(x, y) = (~ + y2 (4y- 3))(x- ~) + t- ~-X: - xH(y- x).
erator. 4.4.3; g(x, y) = -~ cos21r(x- y)- ~sin 21r(x- y)- 4~ sin 21r(y- x).

4.2.9; u(x) = J; g(x, y)f(y)dy-).. J01 g(x, y)u(y)dy+a(1-x)+{3x, where g(x, y) = 4.4.4; g(x, y) = ~ cosxsiny+~ sinxcosy- ~ sinxsin y-2H(y-x) sinxcos y-
x(y- 1) for 0 ~ x<y ~ 1, g(x, y) = g(y, x). 2H(x - y) cos x sin y.
4.2.10; u(x) = fg g(x,y)f(y)dy-)..jg g(x,y)u(y)dywhereg(x,y) = i(x+1)(y-2) 4.4.5; g(x,y) = ~xy- x- -T(x2 + y2) for x < y,g(x,y) = g(y,x).
for 0 ~ x < y, and g(x, y) = g(y, x).
4.4.6; g(x, y) = ~ ln(l-x)+ ~ ln(l +y)+ ~for -1 ::; x < y S 1, g(x, y) = g(y, x).
1
J;
4.2.11; u(x) = f 0 g(x, y)f(y)dy- ).. g(x, y)u(y)dt where g(x, y) = 2~ xn(yn -
y-n) for 0 ~ x < y ~ 1,g(x,y) =g(y,x). 4.4.7; u(x) = l cos2x + ({3- a)~;+ ax-~~ (a+~).
 578 APPENDIX A. SELECTED HINTS AND SOLUTIONS 579
4.4.8; u(x) = -~xcosx + cosx + f2 sin3x- /; sinx. 5.1.9; The Euler-Lagrange equations are y" = z, z" = y.
4.4.9; u(x) = 0.
5.1.10; The Euler-Lagrange equation is ~ - y = 0.
4.5.1; u(x) = aiJ3 - f +ax + 0;a x 2 + E:.:
1 n~:-2 cos mrx, where
J:
bn = -2 (f(x) +a - /3) cos mrxdx.
5.1.12; y(x) = 2sink1rx.

4.5.2; Use Fourier series on [0, 211"]. 5.2.1; Uz = 0 at X= O,l.

4.5.3; Use Fourier series on [0, 1]. 5.2.2; Uzz = 0 and f..l.tUzzz- f..1.2Uz = 0 at X= 0, 1.

4.5.4; No eigenfunction expansion solution exists. 5.2.3; H u is the vertical displacement of the string, then T = J~ pu~dx + !
4.5.5; u(x) =ax+ /3- a1r + E:., muHO, t) + mu~{l, t), U = ~ J~( .;(1 + u!)- 1)dx + fu 2(0, t) + ~u 2 (l, t).
1 Cln cos(2n- 1)~, Then, require puu = f..l./z ~ subject to the boundary conditions
where yl+u;:
_ 8 1-coe(:ln-1)!
Cln - ll'(:ln 1)2 (:ln-1)2/:1-1 • mUtt + ku = f..I.--J!.&...;
yl+u;:
at x = 0, and mUtt + ku = -f..I.--J!.&...; at x = l.
yl+u;:
4.5.6; u(x) = -~(Jx:l- !) - !x. Solution is exact if a+! = 0. 5.2.4; puu = f..I.Uzz on 0 <X< 4and!< X< 1, and mUtt= f..I.Uz(~,t)l;~~~.
4.5.7; u(x) = -~(x) + (b + 4c)L1(x), where L 1(x) = 1- x, and ~(x) =
i(x:l - 4x + 2) are Laguerre polynomials. 5.2.5; Use that T = !ml202 + To:~z 2 sin2 8, U = mgl(1 -cos 8).

4.5.8; Use Hermite polynomials: u(x) = (- &x - ~ -a - f )ez /:l.

2 5.2.6; H = !m:i:2+!k1x2+tk2x4 • Haridlton'sequationsarep = -k 1 q-k:~q3,q =
!!l, where p = m:i:, q = x.
4.5.9; Eigenfunctions are r/Jn(x) =sin~"' (x + 1), ,\ = n
2
t, so an = 0 for n even, 1 • :1 1 •2 • • •2
Cln = - ~ n:~ 4 for n odd. 5.2.7; T = 2mtlt8t + 2(l~81 + 2hl:~ cos(81- 8:~)8t8'J + l~82 ), U = mtltg(1-
cos8t) + m:~g(h(l- cos81) + l'J(l- cos8:~)).
4.5.11; rp!:> = cos(<n;!~b), Ak = 1 ), k = 1, 2, ... , N -1.
-4sin2 :~c:k':
5.2.9; Set x = p+cosp,y = sinp, and then T = mp2(1-sinp),U = mgsinp, so
4.5.12; For ,\ = 411":1, eigenfunctions are 1, cos 21rx, sin 21rx. For ,\ = 411" 2n 2 with that 2ji(l- sinp)- p:l cosp- gcosp = 0.
n > 1, eigenfunctions are cos 21rnx and sin 21rnx. 5.3.1; (a) u(x) = a(1 + ~~x + ~x:l).
4.5.14; With u(x;) = u;, require u;-1 - 2u; + u;H = h:l J::~ r/J;(x)f(x)dx.
1
1
(b) u(x) = a(~:g + ~x + ~~x2 ).

5.1.1; (a) (~ )' = 0. 5.3.2; (a) u(x) = a(1+ ~~~~x+ 2:1~l x2 + ~::!x ) = a(l.O+ 1.013x+0.4255x2 +
9 3

3
0.2797:~: ).
(b) fl-y=- cosx.
(c) y' -y = -ez. (b) u(x) = a(U:i; + :~jWx + ~:~~~x:l + ::6~7 :~: 3 ) = a(0.99997 + 1.013:~: +
0.4255x2 + 0.2797x ).
5.1.2; y(x) = l{x:l- ax+ 1).
W O<x<l
5.1.4; Find the increase in energy due to rotation and determine how this affects 5.3.3; u' = { Ui' 1 < <I .
199' 2 X
the moving velocity. Then determine how this affects the fastest path.
Show that the rotational kinetic energy of a rolling bead is 2/5 that of the 5.3.4; (a) u(x) = 1- x -112x(1- x) = 1- x- 0.230x(1- x).
• T = Jo
1'ts t ranslat'1onal kine t'1c energy. Minimize' J
7 (l+V' 2 > -~-
rzt iog(IIO-II) u;c;.
(b) u(x) = 1-x-f2x(1-x) = 1-x-0.227x(1-x). The exact solution
is u(x) _ elnh(z-1)
1 1 - alnhm •
5.1.6; Maximize f y(x)dx subject to f0
0 vfl + y12 dx =l-a-b, y(O) = a,y(1) =
b. 5.3.5; Using (non-orthogonal) basis functions rp 1 (x) = x, r/J:I(x) = x 2, r/Ja(x) = x 3,
u(x) = a(l + ;r/Jt(x) + /&r/J:I(x) + /forp3(x)). The Sobolev inner product
5.1. 7; The arc of a circle. does not give a useful answer.
 581
580 APPENDIX A. SELECTED HINTS AND SOLUTIONS

5.4.1; Use the functional D(¢) = J01 p(x)¢'2 (x)dx + ap(1)¢2 (1), a 2:: 0, subject (c) iz- il < 2, iz- il > 2.
to H(¢) = J 0
1
w(x)¢ 2 (x)dx = 1 and (d) iz- il < 1,1 < iz- il < v'2, iz- il > J2.
(e) iz- il < 1, 1 < iz- il < v!z, v'2 < iz- il < 2, iz- il > v!z.
(a) ¢(0) = ¢(1) = 0,
(b) ¢(0) = 0, 6.2.6; ~zl=l/2 z21z~l dz = 0.
(c) ¢(0) = 0, and a= 0. 6.2.7; ~zl=l/ exp[z 2 ln(1
2 + z)]dz = 0 (There is a branch point at z = -1).
Show that >.(a) 2:: >.(b)~ >.(c).
6.2.8; ~zl=l/ 2 arcsin zdz = 0 (There are branch points at z = ±1).
5.4.2; Minimize J0 (x¢' 2 (x) + ~ ¢ 2 (x))dx, subject to J; x¢(x)dx = 1, with
1

¢(0) = ¢(1) = 0, or ¢(0) = 0 with ¢(1) unspecified. . r sin z d . h 1

6 ' 2 ' 9 ' Jjzl=1 2z+i Z = 7r Sill 2·
5.4.3; Minimize D(¢) = J;
¢ 12 (x)dx subject to J; ¢ (x)dx =
2
1. Use ¢(x) = 6.2.10; ~zl=l ln~~~ )dz
2
= 0.
v'30x(l- x) to find D(¢) = 10 2:: 1r 2 •
6.2.11; flzl=l cot zdz = 27ri.
5.4.4; The first eigenvalue is approximated by 10, from Problem 5.4.3. The
second eigenfunction is approximated by ¢ 2 (x) = V84ox(x- 1)(x- !), 6.2.13; Hint: Use the transformation z = e
where p = ~ and apply the Phrag-
and D(¢2) = 42. The exact values are .>. 1 = 1r2 , .>. 2 = 471' 2. men-Lindelof theorem to g(O = f(z).

6.1.1; (a) f(-3) = -iVM,f(!) = = -.,fiO.

-ji,J(5) 6.2.14; The function G(z) =
F(z)ei(a+<)z satisfies IG(iy)i :S A and jF(x)l :S
SUP-oo<x<oo IG(x)j. Apply problem 6.2.13.
(b) There are square root branch points at z = ±1 and there is a log-
arithmic branch point on the negative branch of the square root at 6.3.4; !), = R-q
r2-R'
Z -- 13
12'

6.1.2; (e 2 1ri)z is not single valued. .I

g
635
.
• • '
A.
'f'
·.J,_
+ t'f' - a -
2(b -a ) - 4i(b-a)l
3 311"
cz-1)
n z+l

= Az 2 + C~i - i)2, Fx - = -8p1riA.

6.1.3; (a) z =
(b) z
! + 2n7r- i In(2 ± v'3).
= (2n + 1)7r- i In( v'2 + 1), z = 2n7r- i In( J2- 1).
i
:2
6.3.6; (b) w( z)
(c) Fx- iFv = p7r(4rA- 8A i).
2
iFy

/
(c) No such values exist. 6.3.7; The upper half.; plane.
6.1.4; ii = e-(n/ 2+ 2n?r)
for all integer n; ln(l + i}i1r = -7r 2 (t + 2n) + i; ln2; 6.3.8; Flow around a corner with angle(}= {31r. This makes sense only for (3 < 2.
arctanh 1 has no value.
. F ·p - 2 U2 . 2 ia
6 •3 •9 , x - t y - - 1r p a sm ae .
6.1.6; It is not true. For example, consider z 1 = e1ril 2 , z2 = e31ri/ 2 •
6.3.11; f(z) = UJZ2+ a 2 •
6.1.7; The two regions are lzl < 1 and izi > 1; There are branch points at
w = ±1. 6.3.12; Show that 1; = iJ (e-wf 2k + vfl + e-w/k ).
6 • 2 • 1 '. f( Z ) -- 4(z-2)2(z-!)'
15-8i oo dx 271'
6.4.1; J-oo ax2+bx+c = -/4ac-bl'

6.2.2; fc f(z)dz = -27rv'I9(15)11 3 e-i1r/3. roo xsinx dx = .l!:e-lal.

6 •4 •2 .' Jo al+x2 2
6.2.3; The integral is independent of path. fc z- 113 dz = !z 1 i~+~ =
2 3
-3(2) 1f3ei1r/ roo dx - ~.
6.4.3; Jo l+xk - ksin.,.
6.2.4; Find the real part of f(z) = z 112 and use that f is an analytic function.
roo dx - _71'_.
6.4.4; Jo (x+l)xP - sin 1rp
6.2.5; (a) iz- il < v'2, lz- il > v'2.
6.4.5,. Joo x dx = 2J5
iz- il < 2, iz- il > 2.
71' •
(b) 1 (x2+4)-/x'Ll '
 582 APPENDIX A. SELECTED HINTS AND SOLUTIONS 583

6.4.6; fooo i!t dx = 72· 6.5.5; (a) W(Jv, Yv) = ;z.

6.4.7; J~ £~~dx = 211'H(ka)e-k4 ,
(b) W(J.1/t H(t))
II
= 2i11"% •

(c) W(H~ >,H~ >) = -~!·

1 2

6.4.8; Consider f 0 4 :~1. on the rectangular contour with comers at z = ±11'

and z =
±11' - iR, and let R -+ oo. Show that f~w _,. !!~~~ .. dx = 6.5.6; (b) Yn(z) = - ~: r(~~l) + higher order terms.
~(H(a -1)1na -In( a+ 1)).
6.5.8; Use that '.E~=-oo Jn(z)tn = e<t-1/t)z/2.
6.4.9,• Joo .... - coeJi"V•
_ 00 ciJ.Lzdx- w
6.5.13; :E:.0 s2n J~ 1 P~(t)dt = f~ 1 4>2 (z,s)dz =-!In(~:;::>= 2'E:.o 2~~ 1 .
iin\"2- ~(arctanV1-1r).
00
6.4.11; / 0 a:'.tiH =
6.5.23; Period= jifB(~, t) = fif-r(rt~ )·
6.4.12; J:w =
ln(a + bcos9)d8 211'1n(!l±¥). Hint: Differentiate the integral
7.1.2; (a) A with IAI < 1 is residual spectrum, with IAI = 1 is continuous
with respect to b, and evaluate the derivative.)
spectrum, and with IAI > 1 is resolvent spectrum.
6.4.13; / 0
00
.T!&';r;zcix = ! tanh f· (b) Notice that L2 = Li. Then, A with IAI < 1 is residual spectrum,
6.4.14; f0w ln{sin x )dz = -11' In 2. with IAI = 1 is continuous spectrum, and with IAI > 1 is resolvent
spectrum.
6.4.16·' J-oo ~~- dz -- xw(l+wvJ •
00
~ -L
2
(c) An =
~for positive integers n are point spectrum, there is no residual
spectrum since La is self adjoint, and A -:F ~ is resolvent spectrum,
6.4.16,. Joo
-OO
e1'"• ._tl~h
_
a: .J_ _
u;f;- "-1-L1W,
iww A = 0 is continuous spectrum.

roo ln(1+111 2) .J- 7.1.3; Use improper eigenfunctions to show (L 4 - A)- 1 is unbounded. Show that
6 •4 •17; Jo , ...L-2 u;.G = wIn 2• {Xn} with Xn = sin n9 is an improper eigenfunction.
6.4.18; /~00 co:: a:dz = '8
2
2
• 7.1.4; Show that f/>(x) = sinpz is ·an eigenfunction for all p. Notice that the
operator is not self-adjoint.
6.4.20; Evaluate f0 P~(z) on some contour C that contains all the roots.
7.2.1; (a) o(x-~)=2:E~=lsin( 2 n2-l1Tx)sin( 2 n2-l11'~).
6.4.22; The change of variables p2 = r 2 cos2 9+z2 sin2 9 converts this to an integral
{b) o(x- ~) = ~ fo coskxcosk~dk.
00

for which complex variable techniques work nicely.

(c) o(x- ~) = ~ fo sink(x + 4>) sink(~+ l/>)dk where tan</>=~·
00

6.4.23; f0 00 1 2
=.[fir exp(iifi[r).
t- 1 e'"'dt
7.2.6; (a) ~2~42·
6.4.24; J:O tl/2ei"tdt = vrre::!' •.
(b) l!i~.
6.4.27; f~oo (..~:~)1! dz = 2e0111 e-wi/J/'J sin ~IPI/3-t r(1- /3)H( -p). - :1
(c) .j!e=if-.
7.2.7; H(p+1r) -H(p-1r).
6.4.28; Use that V'z+lat!v'z+id = Yif{fa:'fJf"', but be careful to apply Jordan's
1emma correct1y. Then, J_oo00 v'z+at+OI+Vl - ~e f"", -
../i -iw/4e- 110 -jJ
01
-e-""H( P)• 7 • 2 • 8·' ..!.. f 00
211" -oo
-
1
-(iuF(u))e-'":cdu-
l+iiS ,.. ,.. ,.. - J:c roo e:c-•J'(s)ds '

6.6.4; Examine the expression fo" (:l:JE.. ,

make the change of variables x = 7.2.9; Use the convolution theorem to find u(x) = f(x)- t f~oo f(t)e- 31:z:-tldt.
2 2
ycos () + '7sin 6 and then use the beta function to show that f(y) = 7.2.10; f~oo J(e)J(x- e)cie.
~Ju (,~Jjt dz. For another way to solve this problem, see Prob-
Jt ..
lem 7.3.4. 7.2.11; (a) E:z:E~ = ~ which is optimal.
 584 APPENDIX A. SELECTED HINTS AND SOLUTIONS 585

(b) E 111 Ep = ti = 0.274.

2
7.5.5; (a) RL = RRe-2ik<P.
(c) E 111 Ep = !· (b) R};> = R~>e- 2 ik<P.
7.2.12; (a) J~00 eiJ.Iz Jo(ax)clx =f. J:,.. (J:O eilll(p+aaln8)dx) d(J 7 ' 5 ' 6 '. R=
-e2ikta ~~:a cos kaa+ik, sin kaa where k· = .!!!.
ka coa kaa-illt aln ll:~a ' c1 •

= fo~,.. 6(1-4 + asin 8)d8 = Ja;-p2 if II' I < lal and = 0 otherwise. 7.5.7; The gene~al solution is u(z) = aeikz(k 2 + 1 + 3iktanhx- 3tanh~ x) +
(b) Take the inverse Fourier transform of the answer to part (a). ,Be-ikz(k2 + 1- 3iktanhx- 3tanh 2 x). Tr(k) = ~!Z+~H!Z+!~. The bound
states are ¢1(x) = tanhx sechx fork= i 1 and ¢J2(x) = sech2x fork= 2i.
7.2.15; Show that Ne(1) = 1/120, N 6 (2) = 13/60, and N 8 (3) = 11/20. Then
Es( z ) -- wZ
_L ~
eoZ + 20
+ 1l 11 13 +
+ GoZ 1
120.: 2 '

7.3.2; (a) 8-(a+I>r(a + 1).

(b) a.:. provided Re 8 > a. 7.5.13; Use (7.37) to show that cu = 2 ~~:(Li), Ct2 = c~1 = 2~~$~) 1 C22 = - 2A:(}H) ·
7.3.3; u(z) = J; f(y)K(z- y)dy where K = L- 1( +thl,n)·
1
ez/-./2
There is a bound state having f/J(z) = { e-zf../2(1 + ~tanhx), x > 0
z <0

7.3.4; f(t) = linw7r(! 1t f~ Ta- 1T(t- T)dT. at k= ~·

7.3.6; (a) M[H(z)- H(z- 1)] = ~· ~ +bn 1inh~
8.1.2; (a) Vn(x) =an~ sllih , where an= -1i2 f 0b.
am Tg(y)dy,
(b) M[(1 + z)-1] = aln,..n'
(c) M[e- 111 ] = r(3).
bn = ~ J: sin T f(y)dy.

(d) M = 1: , .
(b) u.n (Y) = aalnh(Df{u-11)) ~~~sinh~!: (e)t~e+ aalnh ~
nwalnh li£1 0 a n mrslnh • 11
t
sinh(!!!!:(e-
a
b))F(e)de 1 where Fn({) = !!!!:(!({) - ( -1)ng({)). In the special
(e) M[eilll] = i'r(8). a a

(f) M[cosz] =! COB11'8r(s).

case f(y) =
y(b - y) 1 g(y) =
y2 (y - b), an =-
n'Y',..s (1 + 2( -l)n)),
bn = n1~1 (1- (-1)n). The first representation converges much faster
7.3.7; F(J-4) = f0 00
r/(r)sinJ.4rdr,rf(r) = ~ f0 00
F(J.4)sinJ.4rdJ.4. than the second and gives a good representation of the solution with
only a few terms.
7.3.8; (a) Show that J0
00
Jo(z)dx = 1, and then G(p) = ~·
8.1.3; (a) Require f~oo f~oo z(<f>t + cl/Jz)dxdt = 0 for all test functions f/J(x, t).
(b) G(p) = ~ sinap. e
(b) If U = f(x- ct) 1 make the change of variables =X+ ct, '7 =X- ct
(c) G(p) = sinh- ~· 1 to find J~oo f~oo x(f/Jt + Cl/Jz)dxdt = f~oo f~oo f({)l/J 11 d{d1J =
0 since
f~oo f/Jfld'1 = 0.
7.4.2; Let Un = EJ 9nJIJ where 9nJ = 0, n $ j,gnJ = 11
f_ 1 (J.'n-J- J.'i-n) where
I'~ - >.1-4 + 1 = 0. 8.1.4; u(r,8) = Jrsin8- i-r3 sin38.
cosz,z > 0 { sinz 1 z > 0 8.1.5; G(z,Zo) = - 2~Inl:.:~:_o1 1·
7 •5 •1 i Ut(z)= { coshz,z<O ,u~(z)= sinhz,z<O ·
8 •1 •6 ; u (r, 8) = "' rlnl in8 h 1 r2w (8) e -in8d(J . Reqmre
.Lm,PO an lniRini-1 e , w ere an = 211' Jo g
.
7.5.3; (a) Eigenvalues are>.= -1-42 where tanhJ-4 = -A~,.,J-4 > 0.
I:1f
g(8)d8 = 0.
(b) >.~ = A-1-4 where tan a~=~~ which has positive solutions
if and only if Aa~ > "": . 8.1.8; u(x,y)=En=t
00 ( ~
aniiiilillf£+bn ~h~)
•in cosT, where
(d) C
11 -
_
1 + 2ik(a+P}+agce-
4k
2111
"-1)
I
c _ -e2ika (2i/!k-,1a)+~2iak+,8a)
12 - 41; ' an =- ~?!~ (120( -l)n- 60n 2 ~ + n 4 1r4 + 600),
>.2 = -p where tanh ap = - (a+p)(
p(a+J-+2P)
+JA)+p2. bn = ~?!~ {120 + 600(-l)n- 60n 2 1r2( -l)n + n411'4).
 586
APPENDIX A. SELECTED HINTS AND SOLUTIONS 587
8.1.9; G(x,y;~,r~)
., = ""'oo 1 ..Lsin~sin~e-~lx-el .
L.,.,n= nn- a a
8.1.29; Eigenfunctions are u(r,fJ, <P) = frJm+l/2(J.'mkfi)P~(cosfJ) cosn¢> with
8 110 . ( ) G(
• . , a Joo (
x,y,Xo,yo ) -_ - 21,. _ 00 exp - t'k( X- Xo ))coshk(yo-a)sinhi:Jl.dk
nrnah'-n £Or eigenvalues Amk =( )
~ 2, where Jm+l/ 2(J.tmk) = 0 for m ~ n. Note
y >Yo· (Use the Fourier transform in x.) that J.tot = 1r, J.tu = 4.493, J.t21 = 5. 763, J.to2 = 27r, J.tst = 6.988, etc.
(b) Using Fourier series in y, G(x,y,xo,Yo) = 2::;:'= 1 at exp(-Anlx- 8.2.1; Require h'(x)- ah(x) = f'(x) + af(x).
Xo I) sin Any sin An Yo where An = 2 n±1 ~.
2
8.2.2; Ptt + (>.+ + >.-)Pt = c(A+- A-)P:r: + c2P:r::r:·
• ' u(r ' 8)
8 • 111· = L..,..n=-oo
""'00 anbn
b!ln -a"'2n
an (!.)n-
a (!!)neinll-
r bn (r)n-
b (f!.)neinll
r ' where
an = 2~ I:,. g(8)d8, bn = 2~ Io ,. f(8)d8.
2 8.2.4; G = !H(t- r-lx- W + !H(t- r-lx+W.

8.1.12; If f(O) = l::': 0 ancosn(8- <Pn), then u(r,O) = l::'=oan(~)ncosn(O­ 8.2.7; With appropriately scaled space and time variables, ~-~=sin¢>.
= 2~ Io ,. f(O)dO+ #2:::': 1 (~)n Itr f(<P)cosn(O- </J)d</J.
2
<Pn) This infinite
sum can be summed by converting the cosine to complex exponentials and 8.2.8; u(x, t) = L::'::0 (an sin cyt + bn cos¥) sin T where
using geometric series.
(a) bn = n~~2 sin~,., an = 0,
8.1.13; u(r, 8) = -~ Iooo F(J.L) sina~nlb<~:a~ sin(J.t In fi)dJ.L, where
F(J.L) = fo00
J(Re-t)sinj.ttdt.
(b) an = ni~a sin n211', bn = 0,
(c) bn = 9 ~~2 sin~~, an= 0,
8.1.14; u(r,8) = 2:::'=-ooan (ft)7 sin n:o, where an=~ j ° f(O)sin n:11 dO. (d) an=- c!~~2 (cos n511' -cos~~), bn = 0.
0

8.1.16; Hint: Evaluate L:f=

1 rJ cosj</J using Euler's formula cosj</J = !(eiN +
8.2.9; For a rectangle with sides a and b, w = .f11' = J
! ;b + b. Set A = ab,
e-iJ<P) and geometric series.
8.1.17; Using Mellin transforms,
and find that the minimum is at a = VA.
u(r,O) = 2~ Io e-iklnr kco;hk7r (G(k)sinhk8- kF(k)coshk(B- 1r)) dk
00
8.2.10; For a square of side L, the fundamental eigenvalue is>.= v'2~, whereas
where F(k) = Iooo f(r)eiklnrd;, G(k) =It g(-r)eiklnrdr. for a circle of radius R the fundamental eigenvalue is.\= ~· Take
2
1r R 2 = L 2 and use that ~w = .\. The fundamental frequency for the circle
8.1.18; u(r,8) = 2::'=-oo J:{<:R)einll, where an= ~ 2 I:,. e-inllf(8)d8, and In(x) is smaller than for the square, >.ftrc!e = 0.959>.~Iuare.
is the modified Bessel function of first kind.
8.1.20; u = a2!jj2 (af- /3H(f)).
8.2.12; G(r) = -~H~ )(r) is outgoing as r-+ oo.
1

1
8.2.13; Construct the Green's function from H~ ) (>.jr - el) with A = ~, and
n_ ~-rlnleinll where a = .1.. f 2,. g(O)e-inll df)
00
8 ' 1 •21·' u(r' 0) = ""'
L..,n=-oo ...... ci+lnli3 ' n 2,. Jo • then the solution is proportional to (up to a scalar constant) 1/J(r, 0) =
1 ei>.r sin(>.asln 9) for large r.
8.1.22; -v(~')
_ 8u
= Inn . v~G(x, e)J(x)dVx - Ian n . veG(x, e)v(x)dSx, where 7r >.ainiJ
V - lfn·
8 ' 3 ' 1 .1 C8V _
8t -
1 82 V
R1
V
8z'I" - R2 .
8.1.24; Eigenfunctions are <Pnm(x, y) =sin n:x sin m; 11 , with eigenvalues Anm =
2(n~ m~)
-7r ~+v. 8.3.3; G(x, t; ~, r) = 2L:;:'=1(1- e-n 11'~(t-r)) cosn1rxcosn1r~, fort> r.
2

~
8.1.25; Eigenfunctions are Jn(J.'n!cfi)sinnfJ for n > 0. Thus, eigenvalues are the
same as for the full circle, with n = 0 excluded.
8.3.4; G(x, t) = 2J;te-tf-at.
8.1.26; Eigenfunctions are <Pnmk = Jn(J.'nkfi)einll sin m;z with eigenvalues A = 8.3.5; G(r, t) = (47rt)-nl2 e-r 2
/
4 t, where r = lxl.
I'~ m~
-(1ft-+ (iT). -a. 00 -a.J72 :z::2
8.3.7; T(x, t) =To+ ax+ (Tt- To)7 f_ 00 ~2+1 dTJ, where a= m·
8.1.27; Eigenfunctions are <Pnm(</J,fJ) = P~(cosfJ)sinn</J (or cosn</J) with Anm =
bm(m+l). 2
8.3.8·, X = ln 2 y-w
{ijj = 0.S2ro, t = Inw = 3.47 X 106 s = 40 days.
 589
588 APPENDIX A. SELECTED HINTS AND SOLUTIONS

8.3.9; t = =
At~ In ~ 5.5 hours. At this time the temperature at the bottom
(b) 2Jt = e-Un-1 _ e-Un+l,

of the cup is 116° F. This seems like a long time. What might be wrong 9.2.3; qxt = sinh q.
with the model?
9.3.2; The bound state has J.L = -4· There is a single soliton with amplitude
8.3.10; (a) Require ut =
Di"iJ2u on regions i =
1,2 subject to the conditions !A2 and speed c = 4J.L 2 = A2 .
that u and n · kiVu be continuous at the interface.
9.3.3; Use that r(x) = 6e-x+Bt + 12e- 2x±64 t and find a solution of the form
(b) In a spherical domain, eigenfunctions are
(9.18) with~= x- 4t, 'f1 = 2x- 32t.
lain~
cp(r) = { /. r-h'
0 < r < rp
9.3.4; One soliton will emerge.
ram , rp < r < R
9 • 3 • 5 .' !l£u. i !ku - 0•
dt = - 2k cu' dt -
The tempera! behavior is cp(r)e-1h and the values I" must satisfy the 9.4.3; Verify (9.20) and (9.21).
transcendental equation ~tan~ = --jt; tan e~>.
9.4.4; (a) a= 1 ~~ 2 z- 2i, (3 = 1- a, R = ~·
(c) /-'2 j::::j 6.6 X 10-4 fa.
= _.! (4a5-l~f4a5zl-1) (3 _ 1 zl(16a~-4a5+1)-4afi Th t
(b) a 4 a5z z~-1) ' - 4 a5(zL1) . ere are wo
8.3.11; Solve the transcendental equation Jh:ie-~-ot =(J fort. The velocity is 2 4a
2
rootsof(3(z)=Oatz = ._, ?~ ...
h
t'
8.3.12; Propagation occurs provided h3 (J < /(l) = 0.076, where (c) a= _ ~~
2B (3 = zl±2Bz-1
z2-1 ·

f(y) =
(41ry)- 312 e- 1 / 4 v. The propagation velocity is v where /( lg.) = 9.4.5; anWn = a~z) (z- ~).
h 0, and the minimal velocity is i~·
3

9.4.6; With q0 > 0, set a 0 = !e-qo/2 , a 1 = !eqo/2 , and then use Problem 9.4.4b
8.3.13;
to show that two solitons moving in opposite directions emerge.
u{:c, t) = ei<.lt (1 + iw: ia:-y)eio(L-z) - (1 + iw _ ia:-y)eio(z-L)
9.4.7; Set b1 = -~ =F 0 and then use Problem 9.4.4c to show that one soliton
(1 + IW + ia:-y)eioL _ (1 + iw _ ia-y)e-iaL ,
emerges.
where a2 = 1 + iw. 2 2
9.4.8; Choose spring constants kn with 4k; = 1 + sinb wsech nw.
8.3.14; p(:c, t) = e"-'t P(:c), where P(:c) = exp( -q(w):c), q:l(w) = k+~""'.
10.2.1; En(x) = r(:) L~o(-1)krtltf>.
8.4.1; Un(t) = exp( -e•lniw/k) ) 2) sin(.ar). H we set n and h = we= llf, l, r 1 (cosm + t2)eixtdt = (-i.X
10•2 ' 2·' JO + ~2 + ~
2i )eix _ 2i + i(1 + eix) '\'oo (2!:)2k.
~ L.Jk=O
have in the limit k -+ oo, u(:c,t) = exp(-!jf.)sin(ar), which is the
X X
-
correct solution of the continuous heat equation with periodic initial data. 10.2.3; I; eixtt-lf2dt = Iooo eixtrtf2dt- It eixtt-ll2dt
8.4.2; Un(t) = Jn(-f). _ (;i + ei"' '\'oo (-
1)k-1 r(k±l/2)
- Vx 7/r L.Jk=O xk+t ·

8.4.5; Un(t) = J2n(f). 10.2.4; C(x) = ! A - P(x) cosx 2 + Q(x)sinx 2 , S(x) = h/I- P(x) sinx 2 -
8.4.6; k(w) =cos- 1 (1- ~~~~h 2
). Q(x)cosx 2 , where P(x) = ~ (~- ~ + · · ·),
Q( x) = 12 (lx - ~ 16x + .. ·) .
4x + ~
8.4. 7; For stability, require -2 < 6t.\ < 0, for all eigenvalues ,\ of A.
10.3.1; E1(x) =e-xl:~ 0 (-1)k Jb.
8.4.8; For stability, require that all eigenvalues of A be negative.
roo e-•t )k (4k)l
9.1.3; q(:c) = -2H(:c)sech2 :c. (See Problem 7.5.13.) 10.3.2; Jo l+t' dt = '\'00 (
L.Jk=O -1 z'41'+T.

9.2.2; (a.) !l9f = ian(a~+l -a~).

rl -xtt-ll2dt
10 •3 •5 i Jo e = V z - e-x Yr;;1r '\'00
/Jf 1
L.Jk=O xk+it(l/2-k) ·
 590 APPENDIX A. SELECTED HINTS AND SOLUTIONS 591

10.3.6; f0 00
e:ctt-tclt = y'21jiell (1 - 2 11 ! -
6 ~3112 + · · ·), where 71 = ez- 1. 11.1.3; X1 = 1- E + 3E2 - 12c:3 + 0(E4 ), X2,3 = ±~ -! ± ~iE + !e2 + 0(~).
10.3. 7; I: F sin 1rtdt 2
= ~- 1 2
~~ + (50 - 81r2);; + .. ·. 11.1.4; ¢ + i.,P = U(z- a:)+ iEa(i) 20 + 0(E2).

"00
rff ezt2t-1/3 cos tdt -- 12 L.tlt=O <ct~;
10 • 3 • 8.1 Jo 2
r(lt+tfJ>
z'+t a •
11.1.5; ¢ + i.,P = z + ic:e1z + O(e).
11.1.6; u(x,y) = a(1-y)+by+E(a+(b-a)y)~ -w(x,y)+O(c:2), where V2v = 0
8

10.3.11; (a) I:~=O ( ~ ) kin-" =nIt en(ln(l+z)-z)dx ,.., ~for large n. 2 bz2
and Vz(O,y) = O,vz(1,y) = a/2 + (b- a)y/2, v(x, 0} = a; , v(x, 1) = T·
Use separation of variables to find v.
(b) 2:~=0 ( ~ ) kl..\" = Iooo e-z(1 + ..\x)ndx,.., (n..\)ne-n+! ex:1 >fii· 11.1.8; u(r,O} = rcosO + c:(£{1- r 4 ) + -fi(r2 - r 4 ) cos28) + O(e).

10.3.13; Iooo flee-t Intdt = v'21rxz+te-z lnx( 2zb - 2..;m + ·· ·). 11.1.9; u(r,O) = rcosO- !c:r2 sin29 + O(e2).

1 11.1.11; sfv = c: 2k2/2 + c: 4k4 /4 + O(c:6 ).

10.3.16; ( 1~ ) ,..., 1.82 X 1013 .
• . ; uror a channe1 0 f Wl•dt h 2l , 8
11112 = E2Twk sinh
_,
2
L!l nLI
2kl+2k 2 2
1
nL'ii'I + .. ·.
10.4.1; Io e:~~~;> dx = 21rie-3"14 - 2ie-• l:J!o(-4)i ri~ttfl>. 11.2.1; (a) Eigenvalues are 1, 2, 3, independent of E. Eigenvectors are ( -2, 0,1-
c:}T, (0, 1,0)T, and (0,0, l)T.
10.4.3; I(x) =I: exp[ix(t + t3 /3)]dt = J!"e- 2z/3(1- ~ + 48~\~ 2 + O(x- 3)). (b) The eigenvalue ,\ = 1 has algebraic multiplicity 2, but geometric mul-
tiplicity 1, so the "standard" perturbation method fails. The eigen-
10.4.4; Jn(z) = {£ cos(z- Df- f)- ~{J; sin(z- Df- f)+ O(z- 612 ). values are ..\ = ±~ = 1 ± (c:112 - !c:3/ 2 -}E6/ 2 + 0( c:7/2)), with
corresponding eigenvectors (1,z 2)T where x 2 = ±.j6 = ±(c:1/ 2 +
10.4.5; l'nlc = (2n + 4k- 1)f - 2t,; 2 ~:;!Z~ 1 + O(b ).
1/2c:3/2 + 3/&6/2 + O(e7/2)).
10.4 6· r 1 cos(xt")dt = ,1(.!)1/Prl)- ie 1•
• , Jo ,. z ,. PIC •
11.2.2; y(x) = sinnx + ~(-xsinnx -1rnxcosnx + nx2 cosnx), ,\ = n 2 - E~ +
2 o(e).
10.4.7; Ioff/ {1- ~} 112 cos(:ccosO)dD
=& {ew(~-~(~+iv'ID+···)} -yt~+···. 11.2.3; (b) For X1 sufficiently large (since f(x) has compact support), ,\
e-zt J/(~tofgj)dz
""Ao +
J 0 :» dz
10.4.8; H x > 0, the change of variables s = VZf converts this to !v'zi(x312),
where I(x) is defined in Problem 10.4.3. H :c < 0, Iooo cos(sx + s8 /3)ds"' 11.3.1; Steady solutions are at 8 = 0 and at cosO= ;&r provided 0 2 > f.
2(-~1/4 cos('f +f). 11.3.2; y(x) = c:sinn1rx
2
+ €2(1-coemrz) +2zcoamrz -E2( 2
3~211'2 Srtll*
_ 1)s;nn'/r:C +2E2e\nmrz-n7rZCOinlTZ
~ 8nlirJ cos mr '
10.4.9; Use the answer to Problem 10.4.8 to show that the kth zero of the Airy 2 2
function is at x,
-J( -4k + 1)1r. ..\ = n 1r + t,i;r(1- cosn1r) + 0\~).

oo wdq,...,
-•'1 2
1 3
11.3.3; Bifurcation points occur at solutions of ~(1-/:4) = n 21r2 and there are
10.4.10; (a) f -co v. fl[
Q"(l- ~ + ici2' •• ·), for large a. a finite number of such solutions. The nth solution exists if ~ > n 21r2.
10.5.1; I: f(x)e'"'(:c)dx = f(a)r(~)("'<~ca) )2/3eiltg(a). 11.3.4; u(z, y) = c:sinn1rxsin m;v
+ 0(E2), ,\ = -(n2 + !m2)1r2 - E2a + 0(E4 ). !
6

10.5.4; Set u(x, t) = e-d/2w(x, t) so that Wtt = W:cz + ~w. 11.3.5; u = e¢ + O(c:4 ), ..\ = .\o - ~a4 f::: if a 4 f:. 0 whereas u = c:¢ + O(c:6 ),
11.1.1; X1 = -0.010101, X2,3 = -0.49495 ± 0.86315i. ,\ = Ao - c:4 a5 f<P:dz . a4 = 0, a5 f:. 0.
t/J dz if

= =
11.1.2; x1 0.98979. Since x 0 is a double root of x 3 - x 2 = 0, the implicit 11.3.6; The exact solution is ¢ = 1 + ..\a where a = (1 + .\a) 2. A plot of the
function theorem does not apply for r.oots near x = 0. amplitude a as a function of ,\ is shown in Fig. A.5.
 592 APPENDIX A. SELECTED HINTS AND SOLUTIONS
593

"" . ~w~ cos(¢).

11.4.8; Require a= <I>(¢) = -32 stn
15 :l

. . 10
11.5.1; There are two Hopf bifurcation points, at A= ! ± ~-
11.5.3; Steady state solutions have s = >., x = (>. + a)(l- >.). A Hopf bifurcation
occurs at >. = !(a+ 1).
-o.• -o.2 0.0 0.2 o.•
),
11.5.4; Suppose '1/Jo satisfies AT '1/Jo = 0, and 'l/;1 satisfies AT '1/Jt = -iA'l/; 1 . Then a
small periodic solution takes the form x = e(a¢o + bei>.t¢1 + be-i>.t¢)1 ) +
Figure A.5: Plot of ifJ as a function of A for Problem 11.3.6.
O(e 2) provided 2lbi 2(Q(¢1, ¢}1), '1/Jo) + a(B¢o, '1/Jo) + a 2 (Q(¢o, ¢o), '1/Jo) = 0,
where 2a(Q(¢o, ¢1), 'l/;1) = -(B¢1, 'l/;1).
11.3.7; {a) u(x) = Aa(x), where A satisfies the quadratic equation
12.1.1; u(t, E)= a(1 + ~) cos((l- ~t- E)+ O(e 3).
A2 fol a3(y)dy- ~ +! a(y)dy = 0. fol
12.1.2; u(t) = A( Et) sin( (1 + e2w2)t) + eB( d) cos( (1 + e2w2)t) + 0( e2) where At =
(b) u(x) = Aa(x), where 1 = Asin(A) J; a (y)dy.
2
!A(1- ~A2 ), Bt = !B(1- ~') + {!,(A2 - 4)(6A 2 -1), and w2 = -l
6•

"" a.o 12.1.3; The solution is u(t) = eA(r) sin(t) + O(e 2 ) where AT= !A(1J(r)- ~A 2 ),
and f.L = E21J, T = d.
2.11
15
2.0
-.: 10
~ ...
1.0
c 12.1.4; (a) The Landau equation is AT= -iA 3
, with solution A(r) = ,; 3 ]t~,,,
o.o where A(O) = Ao, so that u(t) = V1+AfA oft sin(t + ¢o) + O(e).
4
01
1 I I I i I :p=:, I 0.0
o.o o.z 0.4 o.e o.a
),
1.0 1..2 1.4 1.1 ... .. -:z 0
),
(b) The Landau equation is AT = - 3~ A 2 , with solution A( r) = , , 1r_Ap~_
3

where A(O) = Ao.

Figure A.6: Left: Plot of A as a function of A for Problem 11.3.7a with a(x) = 12.1.5; Take a= 1 + E2 a 2, take the slow time to beT= E2t and then the solution
sin(x); Right: Plot of A as a function of A for Problem 11.3.7 with a(x) = sin(x). is u(t) = A(r) sin(t + ¢(r) + O(E) where f/JT = !(a2 - 152 + ! cos 2 ¢),
AT = ~sin ¢cos¢.
11.3.0; u(x) = exp(Acosx), where A= Af~1r eAcoau cosydy. 12.1.6; The leading order solution is u(t) = A(r)sin(t + ¢(r)) where r = d and
AT= iA 2 cos(¢), ifJT =~A sin¢.
11.3.10; Solve the system of equations v" +n 2 rr 2 v = e(<P+v) 2 -1'¢1-Jlv, J01 vf/Jdx =
12.1.7; (a) The Landau equation is AT = -~A 3 cos(r), with solution A(r) =
0, I'= 2e J;(q, + v) 2 q,dx, where ¢J =sin nrrx.
V ;A_o , where A(O) = A0. This solution exists for all time
3A 0 smT+4
1
11.4.3; (a) a= 1- 12 e2 + O(e3 ), u(t) = sin(t) + tecostsint + O(e2 ), or a= provided A5 < ~.
6 2
1 + 12 e + O(e3 ), u(t) = cos(t) + je(cos t - 2) + O(e2 ). (b) The Landau equation is AT= -%A 3 q(r).
(b) The two curves are a- 2 - _lll_e 4 +0(e 6 ) and a= 4- .!..e2 +
- 4- .!..e
30 21600 , 30
_!ll_e
21600
4 + O(e 6 )
.
12.1.8; Let 2
T = e t and then u(t) = A(r) sin(t + ¢(r)) where AT = -!A, and
-~.._ 1A2
'/'T- B •
11.4.5; Set u = Aeiwt and find that A 2 ( (b 2 A 2 + 1 - w2 ) 2 + a2 w2 ) = iF 2•
12.1.9; With u = RcosB, then (from higher order averaging) Rt = -\'-R3 - ~R+
2
11.4.6; Set U = a(x)uij(x) and show that for a(x) to be bounded, it must be that O(e3 ). The solution is R 2 (t) = 16e.R5e-• t(16e + 3.R5(1- e-• t)- 1. The
2

f~oo g(x)uti(x)dx = 0. behavior predicted by the leading order equation Rt = -\'-R3 is incorrect,
because it predicts algebraic rather than exponential, decay.
11.4.7; (a) Use the implicit function theorem to show that u = _l+fw~ coswt +
O(e2 ). 2.1.11; The exact solution is u(x) = aJ0 ( ~eax) + bYo( ~eax). The approximate
solution is (set E = %and employ adiabatic invariance, or approximate
(b) Require a= ~(ifJ) = 3
~wsech w21r sinifJ. the Bessel functions for large arguments) is u(x) "'e-ax/ 2 sin( ~eax + ¢).
 595
594 APPENDIX A. SELECTED HINTS AND SOLUTIONS

12.1.12; Write the equation (12.16) as the system u 111 = r(~)v, Vz = J(x, ~),and (b) u(x) = 4::~:-:_13 - 2tanh(z;t + tanh- 1 (i))
then make the exact change of variables u = Uo + EW(~)z, v = z + EZ, u(x) = H(x- i> 51:+h + t(l- H(x- i)) ia:-:!3 - ~ tanh(lf(x -!))
1
(c)
where~= r(u)-r, and Z = J; J:
J(y,u)du-u f(y,u)du, with u = ~· where H(x) is the usual Heaviside function.

12.1.13; Set v = z + Eh(~)u where h' = g- g, and then !!if = z + eh(~)u, ~ = 12.3.8; u(x) = a+~-l + dn(cosh( 2z-<;!.B-l)) + O(e).
-gu- Eh(~)z -·f2h(~) 2 u.
L2.3.11; (a) T"' 2ln 2 ~a.
12.1.15; In dimensionless time, rc has units of length- 2 •
A good dimensionless
parameter is E = ~, r = r c + r m/ h. The function W is a piecewise linear (b) T(v) = -r! 1 ln !~ !+-r :, where c(v) = v':~v!i·
"sawtooth" function with slope - roh?frm •
12.1.16; Write the equation as the system u' = v,v' = u- (1 + g'(~))u8 + Eav.
Then the transformation v = z + Eg( ~) transforms this into a system
to which Melnikov's method can be applied. The requirement is that
=
af~oo U/?(t)clt J~00 9'(!.¥)U8(t)Ub(t)clt, where Uo(t) v'2sech t. =
12.2.2; The device is a hysteretic switch. There are two stable steady outputs,
Vo = V~ if V<V~ and Vo = VR_ if V_ > VR-.

VR+ for v > AvR+

12.2.3; (a) i = "Ri'0 where v0 = i for AVR- < v < AvR+ , where A =
{ VR- for v < AVR-
Bl~'iz;·
(b) The current in the left branch of the circuit satisfies C RpR• ~ +
R.i2 = v- v1 , thus the effective inductance is L = CRpR•.
12.2.5; u(t) = rh + e-t/e + O(E).
12.2.6; u(t) = 1 ~1 + O(E}, v(t) = (t+!>:~ + e-t/'- + O(E).
12.3.2; To leading order in E, u(t) = -ln(~) - ln(2)e-t/ 2 '- '(.~ sin
11
l3ta +
cos~).

12.3.3; u(t) = -tan-1 (t) + E113 exp( 2 ;ih) (l sin( 2~a)- cos(~))
1 1
+ i1 El/322/3 exp(2 / (t-l))
,_1/1

12.3.4; For all {3 there is a solution with u(x) ""'-1+ a boundary layer correction
=
at x 1. For {3 > 0.2753 (the real root of 3x 3 + 12x2 + llx- 4 0}, there =
is a solution with u(x) ""'2+ a boundary layer correction at x = 1.
·12.3.5; The solutions are u(x) = ;h - tanh( 111; [1 ) + O(E) and u(x) = ~ -
tanh( 111 2?) + O(E) with 1/J. and fb appropriately chosen.
12.3.6; The two solutions are u(x) = ::;:~ + 2e-z/e + O(E}, and u(x} = ::~ -
fe-
111
/'- + O(E).

12.3.7; (a) u(x) = ;h- tanh(fe- tanh- 1{j)).

 INDEX 597

Cauchy's equation, 234 LU, 32

Cauchy-Riemann conditions, 215, 355 QR, 33,36
causality, 149, 369, 414 singular value, 37
characteristic polynomial, 11 spectral, 15, 37, 52

Index Chebyshev polynomials, 50, 73, 168,

281
degenerate kernel, 112
delta distribution, 139, 340
chemostat, 503 delta sequences, 134
circulation, 235 dendrites, 409
coaxial cable, 365, 380, 406 difference equation, 170, 324
Abel's equation, 158 Bessel's inequality, 68, 111, 136, 163 commutator, 418 difference operator, 310, 332
Abel's integral equation, 103 beta function, 261, 281 compact differential-difference equations, 390
activation energy, 539 bifurcating solution branch, 479 disc, 300 diffraction pattern, 376, 407
adiabatic invariant, 511 bifurcation point, 479 operator, 111 diffusion coefficient, 380
adjoint matrix, 13 biharmonic equation, 474 support, 87, 137 diffusion-reaction equation, 380, 382,
adjoint operator, 106, 152 bilinear transformation, 237 complementary error function, 442 383, 389
admissible functions, 177 binomial expansion, 70, 279, 443 complete set, 68, 113 digital recording, 300
age structure, 102 biorthogonality, 52 complete space, 61 digital signal, 463
airfoil, 241, 243 Biot-Savart law, 343 complex dilation equation, 80
Airy function, 467 bistable equation, 549 calculus, 214 Dilbert space, 65
algebraic multiplicity, 11, 43, 52, 476 Blasius theorem, 233 conjugate, 209 dimension, 3
amplifier, 523 blood flow, 367 differentiation, 214 dipole, 230
analytic continuation, 227 bound states, 317 functions, 211 dipole distribution, 139
analytic function, 214 boundary conditions integration, 217 dipole source, 342, 358
Argand diagram, 209 Dirichlet, 338 numbers, 209 Dirichlet boundary conditions, 338
Arrhenius rate function, 539 inhomogeneous, 154 velocity, 231 Dirichlet-Jordan convergence theo-
associated Legendre equation, 268 mixed, 338 computer aided tomography, 105 rem, 164
associated Legendre functions, 167 natural, 185, 205 conformal map, 236 discrete Fourier transform, 76
asymptotic sequence, 438 Neumann, 338 conservation law, 380 discrete sine transform, 77, 170
asymptotic series, 437 periodic, 152 constitutive relationship, 380 dispersion, 395
averaging theorem, 512 separated, 147, 152 continuous linear operator, 105 dispersion relation, 463
boundary element method, 358 continuous spectrum, 284, 312 distributions, 138
B-splines, 81, 89, 304 boundarylayer,521 contraction mapping principle, 122, delta, 139, 340
Banach space, 61 boundary operator, 151 313,498 dipole, 139
band limited, 299 bounded linear functional, 108 convergence, 61,437 Heaviside, 139
basis, 3 bounded linear operator, 105 convolution, 172 higher dimensional, 339
Battle-Lemarie function, 305 branch cut, 211 integral, 78 regular, 139
Belousov-Zhabotinsky reaction, 557 branch point, 211 kernel, 296 singular, 139
Bernoulli's law, 234, 243 Bunyakowsky inequality, 6 theorem,294,295,304,356,384 divergence, 228, 234
Bernstein polynomials, 69 cosine integral transform, 290 divergence theorem, 338, 380
Bessel functions, 69, 169 Calm-Allen equation, 382, 549 Courant minimax principle, 19 domain of an operator, 106, 151
modified, 265 Calm-Hilliard equation, 383 cubic splines, 89 dot product, 4
of the first kind, 263 canonical variables, 188 curve fitting, 25 drag, 232
of the second kind, 264 CAT scan, 105 cycloid, 180, 185, 204 dual manifold, 153
of the third kind, 265 Cauchy inequality, 6 Duffing's equation, 194
spherical, 364 Cauchy integral formula, 220 Daubechies wavelets, 82, 307
Bessel's equation, 169, 262 .Cauchy sequence, 61, 68 decompositions eigenfunctions, 161, 283, 359, 362

596
598 INDEX INDEX 599

improper, 296
nodal curves, 361
eigenpair, 11, 161
eigenvalue, 11, 161, 199, 283
eigenvector, 11
elastic membrane, 194
elastic rod, 189
elastica equation, 193
elementary reflector, 34
elementary row operations, 30
entire function, 226
enzyme kinetics, 523
equivalent matrices, 10
essential singularity, 227
Euclidean inner product, 4
Euclidean norm, 5
Euler column, 193
Euler identity, 209
Euler-Lagrange equation, 178
exponential integral, 440
fast Fourier transform, 77
Fermat's problem, 179
Fick's law, 380
filter, 78, 296
finite element space, 88
finite elements, 88
FitzHugh-Nagumo equations, 503
formally self-adjoint, 153
Fourier coefficients, 68, 136, 162
Fourier convergence theorem, 74
Fourier cosine integral transform, 290
Fourier integral transform, 290
Fourier series, 68, 74, 164, 303
Fourier sine integral transform, 289
Fourier transform theorem, 290
Fourier's law of cooling, 380
Frechet derivative, 178
Fredholm alternative, 24, 110, 156,
476, 493, 507, 514, 516
Fredholm integral equation
of the first kind, 101
of the second kind, 102
free boundary problems, 243
Frenet frame, 489
Frenet-Serret equations, 489
Fresnel integrals, 464, 466
Frobenius norm, 41
Fubini's theorem, 64
function spaces, 60
functional, 108, 177
fundamental theorem of algebra, 276
fundamental theorem of calculus, 218
Gabor transform, 301
Galerkin approximation, 92, 124, 131,
176, 197
Galerkin method, 349
Galerkin projections, 399
gamma function, 259, 441
gauge function, 438
Gauss-Seidel iterates, 46
Gaussian elimination, 30
Gegenbauer polynomials, 73, 168
Gel 'fand-Levitan-Marchenko ( GLM)
equation, 413, 415, 433
generalized Green's function, 159
generalized Laguerre functions, 168,
281
generating function
for Bessel functions, 266
for Legendre polynomials, 269,
280
geometric multiplicity, 11, 52, 476
Gram-Schmidt procedure, 8, 34, 39,
65, 68, 114
Green's formula, 338
Green's functions, 146, 284, 339, 383
generalized, 159
modified, 159, 175
Green's theorem, 217
Haar functions, 83
Hamilton's equations, 188
Hamilton's principle, 186
Hamiltonian, 187
Hammerstein integral equations, 500
Hankel functions
of the first kind, 265, 407
of the second kind, 265
Hankel transform, 309, 352
Hardy-Weinberg proportions, 527
heat equation, 338
Heaviside distribution, 139
Heisenberg uncertainty principle, 334
Helmholtz equation, 360
Hermite functions, 89
Hermite polynomials, 50, 73, 169,
298
Hermitian matrix, 14
Hilbert space, 65, 283
Hilbert transform, 355
Hilbert-Schmidt kernel, 103, 112
Hilbert-Schmidt operator, 112, 161
hodograph transformation, 245
Hooke's constant, 189, 319
Hooke's law, 368
Hopf bifurcation, 494, 496, 510
Householder transformation, 34
ill-conditioned matrices, 40
implicit function theorem, 470, 475,
513
improper eigenfunctions, 296
improper eigenvalues, 296
indicia! equation, 263
induced norm, 6
inequality
Bessel, 68
Bunyakowsky, 6
Cauchy, 6
Schwarz, 6
Sobolev, 66
triangle, 5, 60
influence function (see also Green's
function), 146
inner product, 4, 65, 108
inner product space, 4
integral equations
Abel, 103
Fredholm, 101, 102
Hammerstein, 500
singular, 103
Volterra, 101
integration
by parts, 217, 338
complex, 217
contour, 220, 291, 353
Lebesgue, 63
line, 217
Riemann, 63, 217
interpolation, 78, 397
invariant manifold, 14
inverse integral operator, 110
inviscid fluid, 228
irrotational flow, 229
isolated singularities, 226
isoperimetric problem, 181
isospectral flow, 417, 418
iterative solution technique, 121
Jacobi iterates, 45
Jacobi polynomials, 73, 168
jet nozzle, 243, 246
Jordan's lemma, 253, 291, 292, 353,
413, 450, 461
Josephson junction, 406
Joukowski transformation, 242, 311
jump condition, 147, 159
Kaplan matching principle, 530, 541
kinetic energy, 186, 188, 204
Kirchhoff's laws, 365, 485, 523
Korteweg-deVries {KdV) equation,
418
Kronecker delta, 3
Kutta condition, 242
.L'Hopital's rule, 251, 264
lady bugs, 366
Lagrange multiplier, 18, 180, 198
Laguerre polynomials, 50, 73, 168,
298
Landau equation, 509, 593
Langrangian, 187
Laplace transform, 307, 331
Laplace's equation, 223, 229, 337
Laplace's method, 442
Laplacian operator, 337
lattice, 22, 310, 420
Laurent series, 225, 267
least squares problem, 26
least squares pseudo-inverse, 28
least squares solution, 26
600
Lebesgue dominated convergence the-
orem, 64, 94, 134
Lebesgue integration, 63, 94
Legendre function
of the first kind, 269
of the second kind, 269
Legendre polynomials, 9, 50, 72, 167
Legendre's equation, 262, 268
Lewis number, 540
lift, 232
linear combination, 2
linear functional, 108, 137
bounded, 108
linear manifold, 14, 107
linear operator, 105
linear vector space, 2
linearly dependent, 2
linearly independent, 2
Liouville's theorem, 221
Lipschitz continuity, 136, 164
logistic equation, 471
LU decomposition, 30, 32
lung tissue, 409
Lyapunov-Schmidt method, 482
Maple, 48, 50, 453, 455, 492
Mathematica, 48
Mathieu's equation, 502, 552
Matlab, 48
maximum modulus theorem, 222
maximum principle, 17, 115, 117
mean value theorems, 222
Mellin transform, 308, 354
Melnikov function, 494
method of averaging, 511
method of images, 343, 370
method of lines, 391
Meyer functions, 304
Michaelis-Menten law, 525
microwave ovens, 24
midline strain, 191
minimax principle, 19, 199
minimum principle, 199
modified Bessel function, 265, 266
modified Green's function, 159
modulus of continuity, 71
INDEX INDEX 601
moment of inertia, 53 Hermite, 50, 73, 169, 298 reflectionless potential, 323
Moore-Penrose least squares solution, Jacobi, 168 region of influence, 369
28 Laguerre, 50, 73, 168, 298 Reisz representation theorem, 138
Moore-Penrose pseudo-inverse, 28 Legendre, 9, 50, 72, 167 relaxation oscillations, 545
mother wavelet, 80 orthogonality, 7 removable singularity, 226
multigrid method, 4 7 renewal processes, 102
multiplicity Parseval's equality, 69, 113, 271, 293 residue, 225
algebraic, 43, 52, 476 path independence, 219 residue theorem, 225, 285, 414
geometric, 52, 476 pendulum, 148, 188, 510 resolvent kernel, 120, 128
multiscale method, 509 perihelion of Mercury, 183, 483 resolvent operator, 110, 118
periodic boundary conditions, 152 resolvent set, 283
natural basis, 3 Perron Frobenius theorem, 44 resonant frequencies, 24
natural modes, 23 Phragmen-Lindelof theorem, 222 Riemann mapping theorem, 237
natural norm, 6 pitchfork bifurcation, 482 Riesz representation theorejll, 108
near identity transformation, 512 Plancherel's equation, 293 Rodrigue's formula, 268
Neumann boundary conditions, 338 Poisson equation, 337, 392 root cellar, 388
Neumann iterates, 122 Poisson kernel, 223 rotating top, 206
Newton's law, 233 pole of order n, 226 rotational matrix, 12
Newton's method, 131 population dynamics, 101, 472
nodal curves, 361 populations genetics, 526 saddle point, 449
normal equation, 27 positive definite saddle point method, 450
normal modes, 23 matrices, 18 scalar product, 4
normed vector space, 5 operator, 161, 338 scattering transform, 411
norms,5,59 positive matrices, 44 SchrOdinger equation, 169, 313, 411
Euclidean, 5 potatoes, 386 nonlinear, 420
Frobenius, 41 potential energy, 187-189, 194 SchrOdinger operator, 312
induced, 6 potential function, 228 Schwarz inequality, 6
£ 2 ,60 power method, 43 Schwarzschild metric, 182
LP,60 precompact, 111 secular terms, 506, 532
IJ', 59 principal branch, 211 self-adjoint, 13, 153
natural, 6 problem of Procrustes, 41 separable kernel, 112
Sobolev, 66 projection, 8, 98, 119, 124, 347, 397 separated boundary conditions, 147,
supremum, 5, 60 pseudo-inverse, 28 152
uniform, 60 pseudo-resolvent operator, 120 separation of variables, 345
null space, 106 Pythagorean theorem, 7 sequence spaces, 59
sequentially compact, 111
operational amplifier (op-amp}, 521 QR decomposition, 33 set of measure zero, 63, 94, 134, 163
orthogonal complement, 15, 108 quadratic form, 17, 186 shallow water waves, 421
orthogonal functions quasi-steady state solution, 523, 525 Shannon sampling function, 78
associated Legendre, 167 similar matrices, 10
Haar, 83 range, 14 similarity solution, 384
Laguerre, 168 range of an operator, 106 similarity transformation, 10
orthogonal matrices, 33 ranking of teams, 44 similarity variable, 384
orthogonal polynomials, 9, 69 Rayleigh-Ritz technique, 92, 197 sine function, 78, 270, 302
Chebyshev, 50, 73, 168, 281 recurrence formulas, 266 sine integral transform, 289
Gegenbauer, 73, 168 reflection coefficient, 316 sine-Gordon equation, 419
602 INDEX INDEX 603

singular Sturm-Liouville operator, 153, 160, two component scattering, 419

integral equations, 103 163, 198, 500 two-timing method, 509
matrices, 11 successive over-relaxation, 46
value decomposition, 37 superposition principle, 200 uniform norm, 60
values, 38 support, 137 unitary matrices, 33
singularities, 292, 308 supremum norm, 60
essential, 227 swimming protozoan, 473
van der Pol equation, 486, 494, 542
Van Dyke matching principle, 530
isolated, 226 symbolic function, 140
pole of order n, 226 variational principle, 177
removable, 226 Taylor series, 224 vector space, 1
sink, 230 telegrapher's equation, 366, 373, 406, vectors, 1
smoke rings, 488 461 Volterra integral equation, 101
smoothing algorithm, 397 test function, 137, 339 vortex, 230
Snell's law, 132, 180 Toda lattice, 420 vortex filament, 488
Sobolev inequalities, 66 tomography, 103 vorticity, 342, 488
Sobolev inner product, 66 transcendentally small, 439, 540
Watson's lemma, 446
Sobolev norm, 66 transfer function, 296
wave equation, 189, 337
Sobolev space, 65, 66 transform theory, 9
wavelets, 69, 80
solitary pulse, 548 transformations
Battle-Lemarie, 305
solitary wave, 421 bilinear, 237
Daubechies, 82, 307
SOR, 46 hodograph, 245
Haar, 83
source, 230 Householder, 34
hat, 83
span, 3 Joukowski, 242, 311 Meyer, 304
spanning set, 3 near identity, 512
mother, 80
special functions orthogonal, 33
weak convergence, 142
Airy function, 467 similarity, 10
weak formulation, 142, 340
· beta function, 261, 281 unitary, 33
Weierstrass approximation theorem,
gamma function, 259, 441 transforms
69
spectral decomposition, 15, 37, 52 discrete Fourier, 76 Whittaker cardinal function, 78, 270,
spectral representation, 11, 283, 297 discrete sine, 77, 170
300
spectroscopy, 24 Fourier, 290
Wiener-Paley theorem, 299
spectrum Fourier cosine transform, 290 winding number, 490
continuous, 284, 312 Fourier sine integral, 289 windowed Fourier transform, 300
discrete, 283 Gabor, 301 Wronskian, 147, 175, 263
point, 283 Hanke1,309,352,405
residual, 284 Hilbert, 355 X-ray tomography, 103
splines Laplace, 307, 331
B, 81 Mellin, 308, 354 Young's modulus, 192, 368
cubic, 89 scattering, 411
square well potential, 321 windowed Fourier, 300 z-transform, 311
stagnation points, 231 transmission coefficient, 316 Zeldovich number, 540
stationary phase, 456 transpose, 13 zero sequence, 138
Stirling's formula, 444, 453 trefoil knots, 488
strain matrix, 190 triangle inequality, 5
stream function, 474 trigonometric functions, 69, 73
streamlines,230 . tunnel diode, 485