Programming

Udo Ernst
Vorlage: Jan Wiersig, WS2006
Update: Udo Ernst, WS2008
Update: Udo Ernst, WS2009
Update: Udo Ernst, SS2010
Translation: Daniel Harnack, WS2012
Institute for Theoretical Physics
University of Bremen
udo@neuro.uni-bremen.de
February 12, 2013
Contents
I. Programming in Matlab 1
1. Basic Structure of a Computer 3
1.1. Hardware Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2. Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3. Programming Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2. Representation of Numbers in the Computer 5
2.1. Bits and Bytes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2. Representation of Real Numbers and Numerical Errors . . . . . . . . . . . . . 7
2.2.1. Range Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.2. Rounding Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3. Basic Commands and Variables 9
3.1. Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2. Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.3. The Colon-Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.4. Mathematical Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.5. Complex Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.6. Program Control Commands and Function Denition . . . . . . . . . . . . . . 14
3.6.1. Loops (for-Loops) . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.6.2. Conditional Loops (while-Loops) . . . . . . . . . . . . . . . . . . . 16
3.6.3. Bifurcations/Conditional Execution (if-Expressions) . . . . . . . . . 16
3.6.4. Data Input and Output . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.6.5. Example of a Simple Program: Orthogonality of Two Vectors . . . . . 19
3.7. Self-dened Functions and Sub-programs . . . . . . . . . . . . . . . . . . . . 19
3.7.1. Functions as Input Arguments . . . . . . . . . . . . . . . . . . . . . . 21
3.7.2. Recursive Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4. Graphics 23
4.1. Simple Plotting Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.2. Multiple Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.3. Parametric Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.4. Three-dimensional and Color-coded Displays (Images) . . . . . . . . . . . . . 28
5. Systematic Programming 31
5.1. Planning the Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
iii
5.2. Examples of Program Development . . . . . . . . . . . . . . . . . . . . . . . 33
5.2.1. Finding Prime Numbers . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.2.2. Simulation of Diusion . . . . . . . . . . . . . . . . . . . . . . . . . 38
6. Vectorization and Vector Calculus in Matlab 41
6.1. Basic Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.2. Denition of Matrices and Arrays . . . . . . . . . . . . . . . . . . . . . . . . 42
6.3. Simple, predened Vector Operations . . . . . . . . . . . . . . . . . . . . . . 44
6.4. Reshaping, Copying and Duplication of Matrices/Arrays . . . . . . . . . . . . 45
6.4.1. What Size does an Array have? . . . . . . . . . . . . . . . . . . . . . 47
6.5. Example: Vectorization of a Program . . . . . . . . . . . . . . . . . . . . . . 48
II. Numerical Methods in Matlab 51
7. Integration and Dierentiation 53
7.1. Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
7.1.1. Midpoint Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
7.1.2. Trapezoidal Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
7.1.3. Simpsons Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
7.1.4. Integration with Matlab . . . . . . . . . . . . . . . . . . . . . . . . . 57
7.1.5. Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
7.2. Numerical Dierentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
7.2.1. Right-sided Dierentiation . . . . . . . . . . . . . . . . . . . . . . . . 58
7.2.2. Centered Dierentiation . . . . . . . . . . . . . . . . . . . . . . . . . 60
8. Dierential Equations 63
8.1. Ordinary Dierential Equations . . . . . . . . . . . . . . . . . . . . . . . . . 63
8.1.1. Euler Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
8.1.2. Runge-Kutta Method . . . . . . . . . . . . . . . . . . . . . . . . . . 69
8.1.3. Runge-Kutta Method of 4. Order . . . . . . . . . . . . . . . . . . . . 71
8.1.4. Adaptive Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
8.1.5. Matlab Solver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
9. Data Analysis 75
9.1. Fourier Analysis and Filter Operations . . . . . . . . . . . . . . . . . . . . . . 75
9.1.1. Fourier Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . 75
9.1.2. The Convolution Theorem . . . . . . . . . . . . . . . . . . . . . . . . 80
9.2. Probabilities and Distribution Functions . . . . . . . . . . . . . . . . . . . . . 85
9.2.1. Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
9.2.2. Random Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
9.3. Regression Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
9.3.1. General Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
9.3.2. Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
9.3.3. Goodness of the Fit . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
9.3.4. Non-linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Part I.
Programming in Matlab
1
1. Basic Structure of a Computer
1.1. Hardware Architecture
The nowadays generally applied principle, termed Von-Neumann-architecture following its
rst description by John von Neumann in 1946, denes the main components of a computer,
which are illustrated in gure 1.1.
CPU
processing unit
bus unit
control unit
input/output units
memory
Figure 1.1.: Schematic illustration of the Von-Neumann-architecture.
The task of the processing unit consists, as the name suggests, in the processing of the data,
especially the application of arithmetic and logic operations. Among the most important func-
tions of the control unit is the timely coordination of the sequences in the computer. For
this, the control unit has to retrieve commands from the memory, decode them and control
their execution. In todays computers, the processing unit and the control unit are typically
integrated in the same component: the CPU (Central Processing Unit).
Within the computer, data is processed in various hardware components. The transfer of the
data between the components is mediated by the bus unit.
The Memory consists of an numbered array of cells, in which every cell is capable of storing
a small amount of information. This information is stored as a binary number, i.e. sequence
of ones and zeros (yes/no information), in the memory cells.
Input and output units connect the computer with the outside world. Examples for input units
are keyboard, mouse or joystick. Output units are for example screen and printer.
3
1.2. Software
The term Software encompasses both computer programs and the data used by these pro-
grams. We dene three types:
System software, that is needed for the proper functioning of the computer. Among this
is especially the operating system (e.g. Unix, Linux and Windows) and also supplemen-
tary software such as hardware drivers, etc.
Programming languages
Application software (e.g. text editor, browser, ...)
Games!
1.3. Programming Languages
A programming language subserves the function of transmitting commands to the computer
and providing a well dened sequence in which these commands are to be executed. At this
point, we want to distinguish between three generations of programming languages. The rst
generation consists of the machine languages. A machine language is made up of simple
commands that are represented as binary numbers. For every computer type, more precisely
for its CPU, a special code is necessary. The direct programming in a machine language is
nowadays rarely done any more.
The second generation are the assembler languages. Inspite of binary numbers, coding is done
by symbols. An assembler directive is translated 1:1 to a machine command. Thus, also
assembler programs are specic to the type of processor.
The third generation of programming languages is machine independent, i.e. can be executed
on every CPU. These higher programming languages utilize more complex commands and
program structures, allowing for problem-oriented working and reducing possible errors and
lack of clarity to which machine oriented languages are prone. However, also programs written
in such higher programming languages have to be ultimately translated to machine language.
The programs supplying the translation are called compiler respectively interpreter. Examples
for higher programming languages are Pascal, Fortran, C and C++. An interpreter, on the
other hand, interprets and directly executes every command. Here, examples are Basic and
Lisp. Interpreter programs are easier to write, since debugging is more straight forward.
However, the execution of these programs is slower.
The higher programming languages have developed in various directions. Especially for the
natural sciences, an interesting branch are the interactive calculation tools, like for example
Matlab, Maple and Mathematica. These tools facilitate the computer based work on scientic
problems and questions by providing mathematical partial solutions and graphical representa-
tion. It is distinguished between numerical (Matlab) and symbolic (Maple and Mathematica)
calculation tools. The latter are also termed computer-algebra systems.
4
2. Representation of Numbers in the
Computer
2.1. Bits and Bytes
In digital computers, information is represented by discrete states of circuit elements. The
smallest possible unit of information is one bit (binary digit). It can be in two dierent states,
that may be described dierently:
true false
yes no
1 0
.
.
.
A bit is easily realized physically:
current owing current not owing
voltage there voltage gone
magnetized not magnetized
.
.
.
For example, in the communication between integrated circuits, information is traded via
changes in voltage, while the bits on the hard disk are written by magnetization of ferromag-
netic materials. Between electronic musical instruments, bits are exchanged through current
loops (MIDI-Standard). In the stone age of computers, the fties, bits were even once realized
as air bubbles in a liquid medium.
By combining several bits, complex information like numbers, letters (e.g. 8 bit-ASCII-code),
images, color, music, etc, can be represented. Much like in the decimal system, where whole
numbers are made up of the digits 0-9, the same numbers can be written in the so called dual
system by sequences of 0 and 1:
5
dual system decimal system
0 0
1 1
10 2
11 3
100 4
101 5
110 6
111 7
.
.
.
.
.
.
Two further examples for 8-bit numbers:
01010101 = 1 2
0
+ 0 2
1
+ 1 2
2
+ 0 2
3
+ 1 2
4
+ 0 2
5
+ 1 2
6
+ 0 2
7
= 85
10101010 = 0 2
0
+ 1 2
1
+ 0 2
2
+ 1 2
3
+ 0 2
4
+ 1 2
5
+ 0 2
6
+ 1 2
7
= 170
To represent negative numbers, a small trick is necessary: one specic bit codes for the sign
of the number. In an n-bit number, one would for example reserve the n-th it for the sign,
and the remaining n1 bits code, as usual, a binary number z: If the n-th bit is not set, then
the result is by default +z. If the n-th bit is set, the result is (2
n1
) + z. For illustration,
again a small table:
dual system decimal system
01111111 +127
01111110 +126
.
.
.
.
.
.
00000010 +2
00000001 +1
00000000 +0
11111111 -1
11111110 -2
.
.
.
.
.
.
10000010 -126
10000001 -127
10000000 -128
Certain bit lengths have names:
1 Byte = 8 Bit
1 Word = 16 Bit
1 Kilobyte = 1024 Byte
1 Megabyte = 1024 Kilobyte
1 Gigabyte = 1024 Megabyte
1 Terabyte = 1024 Gigabyte
6
2.2. Representation of Real Numbers and Numerical
Errors
Real numbers are, by their nature, analogue quantities. Hence we would expect the handling
of these numbers on digital computers not to be completely problem-free. Present digital
computers usually represent real numbers as oating-point numbers.
oating-point number = mantissa basis
exponent
.
Thereby, the precision, with which the real number can be represented, is determined by the
number of available bits. Simple precision requires 4 Bytes, for double precision 8 Bytes are
needed. The latter is the default conguration in Matlab. The IEEE format of double precision
uses 53 Bits for the mantissa, 11 Bits for the exponent and for the basis the remaining 2.
One Bit of the mantissa respectively the exponent are used for the sign of the quantity. Thus,
the exponent can vary between1024 and +1023. The mantissa always represents a value in
the interval [1, 2[ in the IEEE notation. Here, the 52 Bits are utilized to add up fractions of
exponents of 2. The value of the mantissa yields mantissa=1 +

52
i=1
b
i
2
i
, with b
i
= 1 , if
the i-th bit in the mantissa is set.
2.2.1. Range Error
The maximal range of the oating-point numbers is determined by the number of bits used
to code for the exponent. A typical number for single precision is
2
127
10
38
and for double precision
2
1023
10
308
.
Via application of arithmetic operations on these numbers, the range can be exceeded. The
error occurring in that case is named a range error. As an example we consider the Bohr radius
in SI units
a
0
=
4
0
h
2
m
e
e
2
5.3 10
11
m .
The quantity h is Plancks quantum of action divided by 2. Bohrs radius is in the range of
single precision oating-point numbers. However, the same does not hold for the numerator
4
0
h
2
1.2410
78
KgC
2
m and the denominator m
e
e
2
2.3410
68
KgC
2
. I.e. neither the
numerator nor the denominator can be represented as a single precision oating-point number.
Hence, the calculation of Bohrs radius by the given formula can be problematic. A simple
solution of this problem lies in the use of natural units, such as Bohrs radius, for distances, etc.
An even bigger problem can be illustrated by the calculation of the factorial. The factorial is
dened as
n! = n (n 1) (n 2) . . . 3 2 1 .
7
In Matlab, it can be easily veried by using the function factorial(n), that the factorial for
n > 170 can not be represented, even with double precision numbers. A way out is provided
by the use of logarithms, since the logarithm of a bigger number still gives moderately small
values, e.g. log
10
(10
100
) = 100. It ensues that
ln(n!) = ln(n) + ln(n 1) + . . . + ln(3) + ln(2) + ln(1) .
For bigger n, the evaluation of this expression is, however, to laborious. If n > 30, one is
advised to use Stirlings formula
ln(n!) = nln(n) n +
1
2
ln(2n) + ln
_
1 +
1
12n
+
1
288n
2
+ . . .
_
.
The factorial n! can than be written as the following
n! = mantissa 10
exponent
.
To get the mantissa and the exponent, we form the logarithm to the basis 10 (reminder:
log
10
(x) = ln(x)/ ln(10))
log
10
(n!) = log
10
(mantissa) + exponent .
We now associate the integer part of log
10
(n!) with the exponent. The post-decimal places
are associated with the mantissa, i.e. mantissa = 10
a
with a = log
10
(n!) exponent.
From these examples we learn that range errors can usually be circumvented with a little
creativity.
2.2.2. Rounding Error
Rounding errors stem from the nite precision of the mantissa. The following program illus-
trates this fact:
x = 1;
while (1+x =1)
x = x/2;
end
x = x*2;
One might think that this constitutes an innite loop. To the contrary, the loop will be left
in nite time. The result for double precision is x 2 10
16
(= Matlab variable eps).
eps is the smallest number with 1 + eps > 1, and is the machine accuracy. Rounding
errors of this order of magnitude occur on a regular basis. For example, Matlab calculates
sin 1.2246 10
16
. It shall be mentioned hat the machine accuracy for double precision
is exactly eps = 2
52
, since 52 bits (plus one bit for the sign) are used for the mantissa. This
rounding error might appear to be small and negligible. However, if further calculations are
performed with rounded numbers, the rounding errors can accumulate with each calculation
and grow to a signicant value.
8
3. Basic Commands and Variables
Matlab is an interactive system used for numerical calculations and visualization in almost
every eld of technical and scientic research. Matlab excels in its ease of use, good graphics
and high eciency.
This introduction explains the basics mandatory to perform simple numerical calculations.
First, we use Matlab as a command line interpreter. Entering of single commands will follow a
prompt, here >>. An important command is help. This command returns descriptions and
syntax for functions and commands. For example
help sin
shows information regarding the sine function. Also a link to the detailed documentations can
be found, that can directly be called alternatively with the command doc. Short descriptions
of commands, that match certain search pattern, can be obtained by lookfor. Further useful
commands includequit and exit, with which the Matlab session is terminated.
3.1. Variables
Variables are introduced by assignment and do not have to be declared or dened. Names of
the variables can be chosen freely. The only forbidden things are the use of special characters
and digits at the beginning of the name. A distinction between uppercase and lowercase letters
is not made. Examples:
a = 3; 3 is assigned to a (no output)
a result output
b = -2.5 2, 5 is assigned to b (with output)
Note that Matlab uses a point to separate pre-decimal places and post-decimal places, following
the conventions of the English language.
A major advantage of Matlab in comparison to a conventional programming language is the
possibility to directly work with vectors. This concept shall only be touched briey here, since
it will be extensively treated in a separate chapter:
a = [3;1]; denes a column vector a =
_
3
1
_
a(k) means the vector component a
k
b = [0,3,-4]; denes a row vector

b = (0, 3, 4) .
9
The commas in square brackets can also be replaced by blank spaces.
Matlab cannot only calculate with vectors, but also with matrices. Thus, linear system of
equations, such as
x + 2y = 3
3x + y = 1 ,
can be formulated elegantly:
_
1 2
3 1
__
x
y
_
=
_
3
1
_
respectively
Dx =

b
with the 2 2 matrix
D =
_
1 2
3 1
_
1. line
2. line

1. line 2. line
and the column vectors
x =
_
x
y
_
and

b =
_
3
1
_
.
Here, x is the solution vector being sought. The formal solution of the equation Dx =

b is
given by
x = D
1

b ,
where D
1
denotes the inverse of the matrix D.
Matrices are introduced in Matlab as the following
C = [1,1;0,1]; denes the 2 2 matrix C =
_
1 1
0 1
_
D = [1,2;3,1]; denes the 2 2 matrix D =
_
1 2
3 1
_
.
The single elements C
kl
of a matrix C can be accessed by C(k,l). Here, k denotes the k-th
row and l the l-th column of the matrix. Interestingly, all variables that are used by Matlab
are matrices, which is how the name is derived (Matrix Laboratory).
The following table lists useful commands to manage variables.
10
who List of all used variables
whos List of all used variables with respective size
clear Variable Delete a variable
clear all Delete all variables
load filename Load variable(s) from a le
save filename variable Save variable to a le
! execute a Linux/MSDOS-command
Some specic variables are predened:
ans Last result
pi
i Imaginary unit

1
Inf Innity (z.B. 1/0)
NaN Not-a-Number (e.g. 0/0)
Note that the predened variable can also be overwritten. Sometimes this happens unintend-
edly and leads to unpredictable results.
3.2. Operations
The basic arithmetic operations like addition (+), subtraction (-), multiplication (*), division
(/) and powers () are dened in a natural manner. Examples using the earlier dened variables
are
z = 10-8; assigns the value 2 to z
t = b*[1;2;3]; assigns the scalar product 6 = 0 + 6 12 to t
F = C+D; assigns the matrix to F
_
2 3
3 2
_
=
_
1 1
0 1
__
1 2
3 1
_
to
x = z3; assigns the value 8 = 2
3
to x
3.3. The Colon-Operator
The colon is used in Matlab to generate sequences of numbers, e.g
x=1:10;
generates a row vector x = (1, 2, . . . , 10), which holds the numbers from 1 to 10 in increments
of 1. The increment can be set between the start and the end value:
x=0:0.1:1;
The result is a row vector x = (0, 0.1, . . . , 1).
The colon-operator is especially handy to access dened elements of vectors, e.g.
11
x(3:6)
gives the elements (0.2, 0.3, 0.4, 0.5). When the colon-operator stands on its own, e.g. x(:),
all elements of the vector are referenced. This becomes especially interesting in the case of
matrices. As a case in point we consider the matrix
A =
_
1 1 2
3 0 1
_
,
from which we want to determine the rst and the third column.
A = [1, 1, 2; 3, 0, 1];
a = A(:,1);
b = A(:,3);
The result is a =
_
1
3
_
and

b =
_
2
1
_
.
3.4. Mathematical Functions
Matlab provides a wealth of mathematical and numerical functions. All functions are applied
to the whole vector respectively matrix. The following tables summarize the basic functions.
abs(x) absolute value
sqrt(x) square root
sin(x), cos(x), tan(x) sine, cosine, tangent
atan2(y,x) arc tangent of y/x in the interval [0, 2)
exp(x) exponential function
log(x), log10(x) natural logarithm, logarithm to the basis 10
besselj(n,x) Bessel function n-th order
rem(x,y) rest after division (z.B. rem(10.3,4) = 2.3)
round(x) rounding (z.B. round(3.2) = 3)
floor(x) round down (z.B. oor(3.2) = 3)
ceil(x) round up (z.B. ceil(3.2) = 4)
norm(x) norm (length) of a vector
sum(x) sum over vector elements
prod(x) product over vector elements
max(x), min(x) maximum, minimum of vector elements
length(x) dimension of a vector
mean(x) mean
std(x) standard deviation
12
size(A) dimension of the matrix A
det(A) determinant of the matrix A
rank(A) rank of the matrix A
rand(N) uniformly distributed random numbers between [0, 1). N N matrix
randn(N) Gaussian distributed random numbers with mean = 0 and variance = 1.
3.5. Complex Numbers
An important feature of Matlab is that it provides a variable type for complex numbers. As
long as it is not already used in another context, like a loop, the variable i is predened as the
complex unit. In general, complex numbers can be dened by invoking the command complex
with two arguments, that represent the real- respectively the imaginary part of the number,
e.g.:
i = complex(0, 1);
Computations with complex numbers can be carried out just like with normal, real values.
Hence, the following to lines
c = complex(17, 42);
c = 17+i*42;
give the same results.
In the imaginary plane, the complex number can be represented as a vector with a length amp,
that starts in the origin and points in the direction phi. These quantities can be derived from
the complex numbers through the commands abs and angle. The commands
amp = abs(c)
phi = angle(c)
give the output 45.3100 and 1.1862. Conversely, a complex number can be dened by a
length and an angle in the complex plane:
c = amp*exp(i*phi);
The real and imaginary part of a complex number can then be extracted by the command
real and imag:
real(c), imag(c)
for example returns the output 17 and 42. The command complex can also handle matrices
and arrays as input. Thus
c array = complex(zeros(42, 1), rand(42, 1));
denes a a 42x1-matrix lled with complex numbers, that have a random imaginary part
between 0 and 1.
13
3.6. Program Control Commands and Function
Denition
Programming in Matlab is carried out via script les and function les. The lenames end
in .m, which is why these les are also named M-les. You should not use blank spaces,
umlauts or mathematical operators (e.g. -) in the le names. One should also avoid naming
the les after a build-in Matlab command, since this command will then be overwritten.
Simply put, script and function les contain a list of commands. These commands will then
simply be executed in succession. More complex program structures are realized with loops and
bifurcations. Essentially, there are the following possibilities to control the ow of a program
These constructs are found in every programming language, not only in Matlab.
Figure 3.1.: Dierent possibilities to control the ow of a program, respectively the succession
of commands contained in a program.
14
3.6.1. Loops (for-Loops)
Reoccurring operations are executed within loops. If it is desired to execute a specic set of
commands n-times in a row, the use of a so called for-loop is advisable. As an example, we
want to create a vector p = (p
1
, p
2
, p
3
, p
4
, p
5
) with p
k
= k
2
, where k = 1, 2, 3, 4, 5. The
symbol % induces commentaries:
for k=1:5 % begin of the loop
p(k) = k2; % entries in the row vector
end % end of the loop
The result is a row vector p = (1, 4, 9, 16, 25). Note that frequently, loops in Matlab can be
replaced by the use of the colon-operator, here p = [1:5].2. The latter is denitely prefer-
able, since the execution is faster. In analogy to the colon-operator, an increment dierent
form 1 can be set, e.g.
for k=1:2:5
15
3.6.2. Conditional Loops (while-Loops)
The dierence between the the while-loop and the for-loop, where the number of times which
the body of the loop is executed has to be known beforehand, is that the execution of the
commands in the while-loop is tied to a condition: prior to every iteration this condition is
tested. If the test turn out positive, the loop will be run through. Otherwise the loop will
be left (or not even entered in the rst place), and the commands following the loop will be
executed. Example:
x=5;
while(x>1) % begin of the loop (infinite loop possible)
x=x/2;
end % end of the loop
The result is x = 5/8 = 0.625.
3.6.3. Bifurcations/Conditional Execution (if-Expressions)
Conditional execution of commands is also mediated by if-expressions. The result of the bi-
furcation depends on the result of the result of the if-statement. Example:
if (x>5)
z = 1;
end
Provided the relation x > 5 is fullled, set z = 1. The following operators are available to
construct relations:
> greater
< smaller
>= greater or equal
<= smaller or equal
== equal
= unequal
& logical and
| logical or
logical not
The command if comes with the command else. Example:
if (0 < x & x < 1)
16
status = 1;
else
status = 0;
end
If 0 < x and x < 1, then set status = 1, else set status = 0. The command else thus allows
to react to a unfullled condition. If this shall be coupled to a further condition, the command
elseif (has to be written as one word) can be used.
3.6.4. Data Input and Output
Keyboard and Screen
For input from the keyboard, the command input is provided:
x = input(Input x: );
The command disp displays text and values of variables on the screen:
disp(The value of x is );
disp(x);
Formatted output is achieved by the command fprintf, e.g.
fprintf(The value of x is %g \n,x);
In this connection, %g is a placeholder for the variable x. The symbol \n forces the following
output to begin in a new line. Another Example is
fprintf(x = %6.2f \n,x);
Here, the rst number after the percent symbol sets the minimal total number of digits or
characters that shall be printed. The second number is the desired number of post-decimal
places.
Storage Media
The output to the hard disk or other storage media takes place in a similar way. Prior to
writing to or reading from a le, this le has to be opened. This is done with the command
17
fopen. Example:
>> fp = fopen(example.dat, w);
example.dat is the name of the data being opened and w indicates that we want to write
something into the data (=write). Upon successful execution of the command, Matlab returns
a so called le handle. This can be understood as a pointer when writing into the le, which
tells the computer where the data is supposed to go.
Similar to printing to the screen, data can now be written into the le:
>> fprintf(fp, x = %6.2f \n,x);
Note that the le handle fp is passed as the rst argument to the fprintf-command.
After writing of the data, the le is closed by the command fclose. In the same manner, the
le handle has to be passed to the command:
>> fclose(fp);
The read-in of formatted text works along the same lines; However, when using fopen, the
option r for reading of data (=read) has to be set, and for read-in, the command fscanf
has to be utilized with a suitable format.
Data can also be written and read in a binary format. Here, the data of the computer is written
into a le in its internal binary representation. An advantage of this type of data storage is
its usually low memory demand. A drawback is that the les produced in this manner can
not be opened and the contents inspected by a conventional text editor . In addition, it can
occur that the data transfer between two computers with dierent machine architecture is
hampered, in case they utilize dierent representations of oating-point numbers and integers.
The respective commands to read and write binary data are fread and fwrite. The les are
opened and closed as usual via fopen and fclose.
Additionally, Matlab provides a third possibility to store data: the commands save and load.
The syntax of these commands is
>> load(filename.mat, var1, var2, ..., varN);
and
>> save(filename.mat, var1, var2, ..., varN); .
The variables with the names var1, var2 until varN are saved respectively loaded. The sux
of a Matlab-le should always be .mat. Loss-free compression and system-wide compatibility
are the advantages of this method. Unfortunately, les created by dierent Matlab versions
are sometimes incompatible.
18
Note: if no variable names are specied, all variables that momentarily reside in the working
memory will be saved.
3.6.5. Example of a Simple Program: Orthogonality of Two Vectors
The following script le probes the orthogonality of two vectors a and

b, i.e. it is tested
whether the scalar product of the two vectors is zero. Please try to understand the program
with the help of the given commentaries.
% orthog - program to test the orthogonality of two vectors (3D)
clear all; % delete all variables
% initalize the vectors a and b
a = input(give1.vector:);
b = input(give2.vector:);
% evaluate the scalar product
ab = 0;
for i=1:3
ab = ab+a(i)*b(i);
end
% tell whether the vectors are orthogonal
if (abs(ab) < 1.0e-8)
disp(vectorsareorthogonal);
else
disp(vectorsarenotorthogonal);
fprintf(scalarproduct=%g\n, ab);
end
Matlab provides an own editor, that facilitates the writing of programs. In our case, the script
le would be named orthog.m. Given that the le is in the correct folder, the program can
be invoked with the command orthog.
3.7. Self-dened Functions and Sub-programs
New, own functions can be dened as a function le. In the same way it is possible to
superimpose existing implementations of Matlab-functions with own functions. Assuming that
we want to implement the Gaussian bell-curve
f

(x) =
1

2
e

x
2
2
2
.
For this end we write the following le gauss.m:
19
function y = gauss(x,sigma)
% function to calculate the Gaussian bell-curve
y = exp(-x.2/(2*sigma2))/(sqrt(2*pi)*sigma);
return;
The parameter y is an output parameter, while x and are input parameters. Note that the
respective le name only diers from the name of the function the le denes by the sux
.m. The function gauss is thus to be dened in gauss.m, nowhere else. The self-dened
function gauss will hence be used just like a built-in Matlab-function. For example with
x = -4:0.01:4;
plot(x,gauss(x,1));
the function can be plotted. Even the help functions help gauss and lookfor gauss work.
Printed on the screen are in this case the comment lines preceding the rst command in the
le gauss.m.
Naturally, new functions can also be provided with several output parameters. For this a
vector-valued output parameter is used, which components hold the actual output parameters.
Self-dened functions can also be called within any script or function le. This can be used
to structure programs into a main program and sub-program to increase the clarity and the
readability of the code ( see e.g. g. 3.2). Conveniently, variables dened within a function
are always local, i.e. unknown to other program parts. Hereby, unwanted side eects are
avoided.
Figure 3.2.: Schematic example of the subdivision of a program (Calculation and illustration
of two oscillation modes of a physical system) into a main program and various
sub-programs.
20
3.7.1. Functions as Input Arguments
In many applications, one may want to not only give parameters to sub-programs, but also
functions. The evaluation of the function passed to the sub-program is done with the command
feval. Consider an example of a sub-program which plots a given function in a predened
interval including annotation of the axes, etc:
function plotfunction(f,a,b) % no output parameter
x = a:0.01:b;
y = feval(f,x);
plot(x,y);
xlabel(x);
ylabel(y);
return;
This sub-program is invoked for example by
plotfunction(sin,0,2*pi);
or
plotfunction(@(x)sin(x),0,2*pi);
3.7.2. Recursive Functions
Recursive functions constitute an elegant method to integrate iterative calculations in a simple
function expression, that is capable of solving complex tasks. A recursive function calls itself
during the sequence of execution of its program code. As can easily be guessed, programming
recursive functions involves the risk that a function calls itself over and over again, but never
returns to the calling program. Thus, a suiting termination criterion has to be provided, that
ensures that after a specic iteration, no further recursion is initiated.
As an example we consider a program implementing a nested intervals method to evaluate the
square root of a number x. We begin with an interval in which the sought for square root will
certainly be, thus for example [0, x]. Now we divide the interval in half and check in which
half the square root has to be. This half we take as the new interval and call the function
anew. The procedure is repeated until the interval is smaller than a given numerical accuracy
and the square root is thus determined with sucient precision:
% recursion_eng.m
% ===========
% calculates the square root of a variable x
% using a recursive nested intervals method.
%
% Invoke: x_sqrt = recursion_eng(x);
%
function x_sqrt = recursion_eng(x, x_low , x_high)
21
% no boundaries set? Then set plausible boundaries!
if ~exist(x_low)
x_low = 0;
x_high = x;
end
% Interval bigger than the accuracy of Matlab?
if (abs(x_low.^2-x) > 4*eps)
% yes, find the midpoint of the inteval
x_mid = (x_high+x_low)/2;
if (x_mid^2 > x)
% the square root is in the lower half...
x_sqrt = recursion_eng(x, x_low , x_mid);
else
% the square root is in the upper half...
x_sqrt = recursion_eng(x, x_mid , x_high);
end
else
% ...no, square root is found!
x_sqrt = x_low;
end
22
4. Graphics
4.1. Simple Plotting Functions
In order to represent data graphically, there exist numerous possibilities of visualization. The
simplest realizations are x-y-plots, where data points (x, y) are plotted in a Cartesian coordi-
nate system. Examples:
t = 0:0.01:2*pi;
plot(t, sin(t));
x = 0:0.01:1;
plot(x, x.2);
The result of the last two commands is the graph of the function f(x) = x
2
in the interval
x [0, 1]. Note the point in front of the power , which ensures that the operation is
performed element-wise.
You can even omit the last argument of plot, in which case one vector with N single function
values vector will be plotted dependent on the set of indices 1, 2, . . . , N:
plot(vector);
Generally, one would avoid the short syntax for small test programs and rather give the x-axis
explicitly.
The correct depiction of a function also needs clear annotation of the axes with the physical
units of the plotted quantities. The x-axis is annotated with the command xlabel(string);,
the y-axis with ylabel(string);. As the name suggests, string is a string, i.e. a sequence
of characters , special characters and numbers, that has to be given in between quotation
marks. By title(string);, the graph can be given a title. Examples for this are:
t = 0:pi/120:2*pi;
plot(t, sin(t));
xlabel(angle [rad]);
ylabel(amplitude [V]);
title(The sine is a pretty boring function);
If one desires to plot more than one function into the same graph, Matlab can be advised
to do so by the keyword hold on; after plotting the rst function. This command prevents
Matlab from drawing new axes for the following plots. All further plotting commands now add
functions to the already existing graphs, until the command hold off; is given:
plot(t, sin(t));
23
Figure 4.1.: Simple example plot.
hold on;
plot(t, cos(t).*sin(t), r);
...
(further plotting commands)
...
hold off;
When several function are plotted into the same graph, optical means of discrimination between
the functions is important. The main possibilities are changing the line style (continuous,
dotted, dashed, ...) or the color of the plot symbols. Matlab accepts these options as a
combined string following the function values as a further argument in the plotting command.
Some possibilities of constructing this sting are the following:
Colors: k, r, g, m, y,...
Line styles: -, , ., -.,...
Plot symbols: o, x, ., d,...
As an example we consider the addition of a cosine function as a violet, dashed line to the
previous example plot:
plot(t, cos(t), m--);
Since multiple annotation of an axis is not possible, a legend can be produced when several
functions are displayed in one graph. The command syntax for this is
legend(string1 string2 ... );
An example for the three functions in the previous example plot is:
legend(sine sine * cosine cosine);
Among further options, extensively listed by help plot;, are the manipulation of the line
width and the size of plotting symbols. These have to be give as a pair of arguments to
24
Figure 4.2.: Three functions in one graph.
the plot function: rst a string, that unambiguously states which plot parameter is to be
set, and second the value that is to be assigned to this parameter. The line width is set via
LineWidth, and the string addressing the size of plot symbols is MarkerSize. Assuming
we would also like to depict the exponential function by a yellow continuous line of line width
4 and circles of the size 10 as plot symbols, we would have to write:
hold on;
plot(t, exp(-t), yo-, LineWidth, 4, MarkerSize, 10);
hold off;
The intervals of the axes shown in the graph can be set by axis([x min x max y min
y max]);. The option tight tells axis to choose the intervals such that the plot win-
dow circumscribes the plotted functions as close as possible. axis off; hides the axes of the
plot. Examples:
axis tight; axis([-5 15 -2 2]);
Did the function turn out nicely and is sought to be saved, the currently active plot window
can be written to a pixel or vector graphics le by print. The syntax for this command is
print(string, dFORMAT, -rRESOLUTION);. Here, string should be replaced by a
le name (without sux, Matlab will append it automatically!), FORMAT by the desired format
and RESOLUTION by the desired resolution in pixel per inch. Examples for supported formats
(all others can be found by the help function) are:
FORMAT: ps, psc, eps, epsc, ti, jpeg, png,...
4.2. Multiple Graphs
Multiple functions do not have to be plotted in the same graph, Matlab also allows to open
several plot windows. A new plot window is created by figure. The windows are numbered
by Matlab. Thus, if one wants to access a specic plot window n (for example to add a graph
25
Figure 4.3.: Example of the denition of two plotting region in a 3x4 raster.
or annotation), the window is addressed by figure(n);. If the designated window does not
exist yet, a new window will be opened.
A window can be closed by close(n);; close all; closes all windows at the same time.
Furthermore the possibility exist to position several graphs next to each other in the same
plot window. For this, Matlab internally subdivides the window into nx times ny rectangles,
that are numbered in ascending order from 1 to nx*ny from left to right and from top to
bottom. A new graph is created in one such rectangle by the command subplot(ny, nx,
k);. The output of the following plotting command will then be shown in the rectangle with
number k. It is also possible to extend graphs across several rectangles that together form a
bigger rectangle. To achieve this, a vector with two components has to be given as the third
argument to subplot, which holds the index of the upper left rectangle and the one of the
lower right of the area where the graph is supposed to go.
Other useful functions are:
grid on, grid off;: activate/deactivate grid in the background of the current plot
loglog(x, y);, semilogx(x, y);, semilogy(x, y);: plotting in logarithmic resp. semilog-
arithmic representation
polar(angle, radius);: plotting in polar coordinates (important for polar bears and pen-
guins!).
26
1 0.5 0 0.5 1
1
0.5
0
0.5
1
f
x
(t)
f
y
(
t
)
Figure 4.4.: Parametric function.
4.3. Parametric Functions
Parametric functions are also easily plotted. First, an independent variable, on which com-
ponents of the function depend, has to be dened as a suciently nely discretized vector.
Second, the components of the function have to be calculated in dependency on this quantity
and are plotted against it.
An example is the harmonic oscillator in two dimensions (2D pendulum), that, due to friction,
slowly comes to a halt (note that Matlab, when creating annotations, even understands a bit
of TeX, which comes in handy for example to set indices or Greek symbols!):
t = 0:0.1:100;
fx = exp(-t/20).*sin(t);
fy = exp(-t/20).*cos(t);
plot(fx, fy);
xlabel(f x(t));
ylabel(f y(t));
You can also animate plots nicely by drawing a sequence of graphs in quick succession. Atten-
tion! Following every complete plotting command, you should then use the directive drawnow;,
to make Matlab display the plot immediately. Otherwise, Matlab will hold back the display
of the plot until the command interpreter is no longer busy and waits for new user input. The
plot that then appears will thus only be the last of the intended animation...
Example, based on the previous function denition:
for i=1:numel(t);
plot(fx(i), fy(i), ko);
axis([-1 1 -1 1]);
drawnow;
end
27
4.4. Three-dimensional and Color-coded Displays
(Images)
Three-dimensional trajectories can be drawn with the plot3-command:
plot3(fx, fy, t);
Two-dimensional functions or surfaces can be depicted as a mesh grid:
[x grid, y grid] = meshgrid(-20:20, -20:20);
figure(1);
mesh(x grid);
title(slanted plane);
axis off;
z = exp(-(x grid.^2+y grid.^2)/100);
figure(2); mesh(z);
title(bell-curve);
axis off;
Figure 4.5.: Examples for grid functions.
A useful command is set(gca, DataAspectRatio, [1 1 1]);, which scales all axes
such that their size relationship on the screen is approximately equal.
The command imagesc(array); depicts the two-dimensional function or matrix array as
a color-coded top-view. The color code can be displayed next to the plot by the additional
command colorbar;:
imagesc(z);
title(Bell-curve, top-view);
colorbar;
axis off;
28
imagesc is a handy tool to display images, that are saved as arrays. If the array has a third
dimension of size 3, the entries of the array will be interpreted as color components according
to the Red-Green-Blue (RGB) system. Attention: the y-axis will be reversed, which can be
xed with the command set(gca, YDir, normal).
Instead of a color-coded illustration also contour lines can be used. The correspondent com-
mand is contour(array);. Optionally, in a second argument the number of contour lines
used for the plot can be set:
Figure 4.6.: Examples for a top-view and a contour plot.
contour(z, 100);
title(bell-curve, contour); axis off;
The following table summarizes the most important graphics commands.
plot(x, y) graph of the vectors x and y
loglog(x, y) logarithmic depiction
semilogx(x,y), semilogy(x,y) semi-logarithmic depiction
polar(theta, rho) graph in polar coordinates
contour(z) 2D contour line plot of matrix z
mesh(z) 3D grid plot of matrix z
title(text) set gure title
xlabel(x),ylabel(y) annotation of the axes
grid on/off grid lines on/o
legend(text1, text2,...) legend
axis([xmin, xmax, ymin, ymax]) scaling of the x- and y-axis
subplot(m, n, p) division in mn subplots
clf delete current plotting window
figure(n) create plot with number n
hold on/off keep/let go of previous graph
print(filename) print/save gure
29
5. Systematic Programming
Computers are at the same time incredibly fast and incredibly stupid. They will do exactly
what they were told, but usually not what you want. To change this or even prevent it
from happening, second to the thorough knowledge of the syntax of a programming language
and the available commands, a careful planning is mandatory this becomes more and more
important the bigger the programs and the more complex the problems in question become.
Simply put, programming is similar to building a model with Lego bricks. From small pieces
something is put together, the nal appearance of which has a wholly dierent character than
the pieces it is made from. In analogy to building Lego models, one has to think about which
pieces to use st, and which to use later... and of course it has to be the correct pieces!
In the following, the systematic path form a problem to the nal program is outlined. Two
examples, nding prime numbers and simulating a diusion process, complete this chapter.
5.1. Planning the Program
The following steps are reasonable to plan and nally write a program.
(a) Problem denition and solution
The rst step is the exact denition of the program, the computer is supposed to solve.
It should be claried, which quantities constitute the input the computer has to process
and which output quantities the computer should return. Also one should be aware by
which means or methods the problem shall be solved (In most cases, an arbitrary amount
of thought and time can be used up by this step. However, for this course, this aspect is
regarded secondary, since in most given problems the ansatz for the solution is already
lined out. We want to focus on programming per se).
As a simple Example we consider the simple conversion of a monetary amount from Euro
to US-Dollar. Input quantities are the amount in Euro as well as the exchange rate. The
output quantity is the amount in US-Dollar. This simple problem of conversion can be
solved by trivial multiplication.
(b) Division into sub-steps
In the second step, the (calculation-)problem is divided into individual and clearly ar-
ranged sub-steps. These sub-steps should be chosen and organized according to the
following aspects:
- Which steps have to be executed rst, which later?
31
The order of commands in a program is always important. For example, before
executing a command, all variables the command uses need to be known and a
valid value assigned to them. Complex calculation on partial solutions can only be
carried out if the partial solutions were computed before. In our simple case, the
amount in Dollar can only be calculated, if the exchange rate has been assigned to
a specic variable beforehand.
- Which steps have to be iterated several times resp. applied in a similar
manner on dierent quantities?
If steps are to be executed several times, loops are a good solution.
If similar tasks occur at dierent places during the solution of the problem, it is
advisable to use a function in the program code.
- Which steps have to be carried out dependent on a specic condition?
Always when a distinction of cases is due, program code will be executed only
under certain conditions. For example the absolute value of a number x is found
by multiplying it with 1 but only if x is negative! Another example: the e-
mail client on your computer will only retrieve emails from a mail-server in case
they do not already exist as local copies on your hard drive (avoiding unnecessary
operation).
- Which steps demand user interaction or an informative feedback?
Some steps might need the conrmation of the user prior to their execution. For
example a program that formats your hard disk should ask you before it starts
running.
Feedback is also important if errors occur during the execution of the program
for example when dividing a number by 0.
- Make sure that the solution can be found in nite steps.
This is also a very important point for the success of the problem solution: The
computer should be done after a nite number of steps, ergo in nite time. If this
is not given, a dierent solution procedure has to be found.
(c) Creating a ow chart (optional)
In addition to the division in sub-steps, you can also create a ow chart of your program,
where the individual sub-steps are connected by directed arrows.
Flow charts may be very helpful to review more complex program logics. Often problem
become apparent when creating a ow chart, that you were not aware of when mentally
dividing into sub-steps. For the creation of this diagrams, even a DIN norm exists (DIN
66001). In the following an illustration will be used that is close to this norm and uses
the ow control elements and boxes introduced in the previous chapter(s).
(d) Transformation of sub-steps into program commands
Only now the carefully structured problem is translated to a specic programming lan-
32
guage. Here, also some guidelines exist that should be adhered to:
- How are data, data structures and intermediary results represented?
Not every mathematical algorithm is portable 1:1 to the computer. Complex num-
bers, for example are unknown to many programming languages these will be
represented by a real part and an imaginary part in two oating-point numbers.
When working with Matlab is is important to consider when to use scalar quan-
tities, vectors, two dimensional arrays or higher dimensional arrays. Important is
also which type of data is used. A oating-point number for example is unsuitable
to represent a string. Also one should always be aware of the memory consumption
of the program.
- Replace every sub-step by at least one program command
- Unexpectedly complex sub-steps should be further broken down
- Adhere to the syntax of commands and expressions
For this, it is mandatory to use the help function! especially for beginners or
when changing to another programming language. Every command has to be
unambiguous, so the computer can understand what you want it to do.
After nalizing your program, it needs to be tested: Does the code perform as intended? If not,
that is it back to the drawing board: First, all the syntax errors are eliminated (somewhere
is a comma instead of a dot, in a comparison the double equality sign was not used, an
intermediate result was not assigned to a variable, etc), and then the logic of the program is
checked are the results plausible?
Assistance for this step the so called debugging provides the debugger, which is part of
many programming languages. Matlab also oers a debugging functions, however, many errors
can be solved by command line based interaction with the (with an error message) aborting
program. A good idea is to test the program with a reduced data set or problem, for example
to run a search for prime numbers rst until n = 17 before making the program return all
prime numbers between one and one trillion.
A lot of the aforementioned steps are not always clearly separable in practice. Most likely,
while breaking up a problem in smaller steps, you will already think about useful commands
you know and how these might help. Program development is thus a constant interaction
between these dierent levels of planning.
5.2. Examples of Program Development
5.2.1. Finding Prime Numbers
The task our example program has to solve is the calculation of all prime numbers in the
interval from 1 to N. Reminder: a prime number is a natural number that is only divisible
without remainder by itself and by 1. 1 itself is no prime number.
33
Figure 5.1.: Flow chart of a brute-force prime number nder
The simplest method to determine whether i is a prime number is the following: One tries
every divisor from 2 to i 1. If a divisor is found that divides without remainder, then i is no
prime number. This so called brute-force-ansatz (leads to an end, but is not very smart) can
be easily implemented in a program. We check, in a loop, whether for a particular number
i a divisor can be found. If this is the case, a so-called ag is set (a variable, that is either
0 or 1 and signal, whether a certain event occurred). If the ag is not set after the loop is
nished, no divisor has been found and the number is remembered as a prime number. If the
ag is not set, the program does nothing. After this, the next number i +1 is scrutinized and
so forth until all natural numbers until N have been tested. The latter will also be done in a
loop. Figure 5.1 shows the ow chart, the following gives the correspondent Matlab code:
34
%%% TASK:
%%% ========
%%% find all prime numbers between 1 and N and save them
%%% in a vector prime_numbers
%%%
%%% solution : brute force
%%% until which natural number shall be calculated?
n = 10000;
%%% there are maximally m<=n-1 prime numbers for all natural
%%% numbers until n
prime_numbers = zeros([1 n-1]);
%%% The next prime number shall be saved in the vector component
%%% with the index idx_prime_number
idx_prime_number = 1;
%%% check all numbers
for i=2:n
%%% try all divisors from 2 to i-1
found = 0;
for divisor =2:i-1
%%% is the number i divisable by divisor without remainder?
if (i/divisor == fix(i/divisor ))
found = 1;
%%% terminate the loop , one divisor is enough!
break; %%%WITH OPTIMIZATION
end
end
%%% No divisor found? Then the number i is a prime number!
if (gefunden == 0)
%%% remember prime number , go to next index
prime_numbers(idx_prime_number) = i;
idx_prime_number = idx_prime_number+1;
end
end
%%% shorten the result vector ,
%%% only keep the found prime numbers
prime_numbers = prime_numbers(1: idx_prime_number -1);
35
Figure 5.2.: Flow chart for the Sieve of Eratosthenes.
For large N this code will run for a long time, that is to say (running Matlab 6.1, Intel DualCore
processor, n = 10000) approximately 141 seconds. When scrutinizing the program code, a
point can be found at which the code can be optimized: As soon as one divisor is found, it
does not make any sense to further execute the inner loop. It can be left with break. This is
especially sensible for large i: If i is already divisible by 2, the divisors from 3 to i 1 do not
have to be tested any more! By this, the execution time of the program reduces itself to 16
seconds.
36
A much more sophisticated method to nd prime numbers is the Sieve of Eratosthenes. The
principle is easy to grasp: First a list of all natural numbers is generated from 1 bis n, from
which successively all non-prime numbers are deleted. Except for 1 at rst all numbers are
candidates form prime numbers. Now every number i from 2 to n is checked. If i is already
marked as a non-prime number, it is skipped. If it is not marked (like for example 2 at the
beginning of the procedure), all multiples of this number are marked as non-prime numbers,
i.e. 2i, 3i, 4i until n. Left over will only be the prime numbers!
See the ow chart of the correspondent program (gure 5.3) and the Matlab code:
%%% TASK:
%%% ========
%%% find all prime numbers between 1 and N and save them
%%% in a vector prime_numbers
%%%
%%% solution : sieve of Eratosthenes
%%% until which natural number shall be calculated?
n = 10000;
%%% define so called flags for all natural numbers
%%% until n. Is the flag equal to 1 at the end of the program
%%% the number is a prime number
flag_prime_number = ones([1 n]);
%%% by definitioin , 1 is not a prime number
flag_prime_number(1) = 0;
for i=2:n
%%% is the current number i a prime number?
if (flag_prime_number(i) == 1)
%%% yes , mark all multiples as non-prime numbers
for j=i+i:i:n % WITHOUT OPTIMIZATION
flag_prime_number(j) = 0; % WITHOUT OPTIMIZATION
end % WITHOUT OPTIMIZATION
%%% flag_prime_number(i+i:i:n) = 0; % WITH OPTIMIZATION
end
end
%%% find the indices of all entries =1 in the vector
%%% the indices are the sought prime numbers
prime_numbers = find(flag_prime_number);
The run time underlines the ingenuity of the sieve-idea: 0.22 seconds instead of 16 seconds!
Additionally, the program code can even be optimized by replacing the inner loop by a vector-
37
ized command (comment all lines marked with WITHOUT OPTIMIZATION and uncomment all
lines marked with WITH OPTIMIZATION): 0.05 seconds!
With this, in comparison to the rst attempt at this program, the execution time could be
shortened by a factor of 6400! Important to note is that both programs are completely correct
and yield identical results, depite their dierent appearance and run time.
5.2.2. Simulation of Diusion
A second example is a simulation of a so-called random walk. A random walk describes a
movement of a particle (e.g. molecule, protein) that is determined by a random process. In
our case the particle starts at the location x = 0 and moves in every time step by a distance of
dx either to the left or to the right. Both possibilities have the same probability of happening.
The task for the computer is to simulate n = 1000 trials with the particle and stop the
simulation each time the particle crosses a barrier at x = 1 or x = 1. The output of our
program shall be the mean number of steps needed to reach one barrier, calculated from the
n iterations.
This is the ow chart (gure 5.3) and the program code:
%%% TASK:
%%% ========
%%% perform a random walk with step width dx starting at
%%% x=0 until x crosses either -1 or +1
%%%
%%% perform this many trials:
n_trials = 1000;
%%% elementary step width
dx = 0.5;
%%% initialize result vector
steps = zeros([1 n_trials ]);
for i=1: n_trials
%%% start random walk at x=0
x = 0;
%%% initialize counting variable
n_steps = 0;
%%% perform random walk until +1 or -1 is reached
while abs(x) <= 1
%%% draw random number.
%%% choose direction with equal probability
38
if (rand >= 0.5)
%%% to the left...
direction = -1;
else
%%% to the right...
direction = 1;
end
%%% update position and number of steps
x = x+direction*dx;
n_steps = n_steps +1;
end
%%% return number of steps
fprintf ([Inthe%i-thtrial ,%istepsneeded ...
untilboundary !\n], i, n_steps );
%%% remember number of steps for trial i
steps(i) = n_steps;
end
%%% evaluate the mean of the steps and return the value
fprintf(themeannumberofstepsusedis%f...\n, ...
sum(steps)/ n_trials );
39
Figure 5.3.: Flow chart for the simulation of a diusion process.
40
6. Vectorization and Vector Calculus in
Matlab
A great advantage of Matlab is that all variables are matrices (or, more general, arrays).
The benets of this principle are manifold: They encompass the simplicity of the syntax to
formulate complex mathematical operations, avoidance of tedious for-loops and optimization
of internal processing of huge amounts of data resulting in considerable acceleration of the
program.
6.1. Basic Syntax
The easiest denition of vectors and matrices is to put the elements of a matrix between
square brackets. Column vectors are treated in Matlab as matrices of dimension N 1 ,
row vectors as matrices of dimension 1 N and scalars (i.e. plain numbers) as matrices of
dimension 11.Rows are divided from other rows by the use of a semi-colon, and columns are
separated by commata (oder simply by a blank space, as in the last of the following examples).
Examples:
scalar: a scalar = 1; 1x1-matrix
column vector: a column = [1; 2; 3]; 3x1-matrix
row vector: a row = [1, 2, 3]; 1x3-matrix
matrix: a fullmatrix = [1 2 3; 4 5 6; 7 8 9]; 3x3-matrix
The same rules that subsume single elements or variables to a vector also apply for strings and
so-called cell-arrays:
b = [A character string of four elements];
c = rand([4 4]) 1:17 complex(42) this is a cell of a cell-array!;
For the ones interested to dive deeper into Matlab: Note that cell-arrays are extremely useful
to save several aspects of data in the same variable. An example from the laboratory would
be a variable that holds the measured data, a date for the day of recording, a description of
the experimental setup in a string and the parameters of recording. Furthermore, structures
exist, whose constituent parts have names that can be accessed with a syntax similar to the
programming language C.
41
6.2. Denition of Matrices and Arrays
Simple vectors can be dened with the already mentioned colon-operator. The syntax is start
value:step width:end value. If the step width is omitted, the default step width of 1 is
used. Examples:
1:7 creates a vector with the numbers 1, 2, 3, 4, 5, 6, 7.
1:2:7 creates a vector with the numbers 1, 3, 5, 7.
4:-0.3:2.8 creates a vector with the numbers 4.0, 3.7, 3.4, 3.1, 2.8.
The vectors dened in this way shall be assigned to a variable:
p = 0:0.01:100;
Such vectors, if used in loops, will be processed element-wise. In the following example, a
vector with the numbers 1, 17 and 42 is printed to the screen element-wise:
to process = [1 17 42];
for k=to process;
fprintf(k is %i\n, k);
end
The output is:
k is 1
k is 17
k is 42
Furthermore, the colon operator can be used to access columns, rows or arbitrary dimensions
of an array. The colon by itself selects all elements of the respective dimension. In the syntax
introduced above, the colon operator can also be used to select a specic set of indices of a
matrix:
b = a(4, 7:5:17); selects, from the 4. row of the matrix a the entries of the columns 7, 5
and 17. It creates a vector from these elements and assigns it to the variable b.
Bigger matrices and arrays are dened by the commands zeros, ones and eye. The two
arguments of these functions assign the number of columns and rows, e.g. fullofzeros =
zeros(17, 42);. zeros lls the created matrix with zeros, ones with ones, and eye creates
an identity matrix. More than two arguments result in multidimensional arrays (tensors).
There are several shortened notations of these commands. Some examples:
z = zeros([1 17]); correct, creates 1x17-matrix.
z = zeros(1, 17); correct, creates 1x17-matrix.
z = zeros([1, 17]) correct, creates 1x17-matrix.
z = zeros(1:17) wrong, tries to create a 1x2x3x4x5x6x7x8x9x10x11x12x13x14x15x16x17-
matrix and does not nd enough memory!
A similar problem can occur when invoking the commands with only one argument n. Matlab
will try to create a nxn-Matrix instead of a 1xn-vector (which might have been intended).
42
z = zeros(1e6); wrong, tries to create a 1000000x1000000-matrix!
z = zeros(1, 1e6); correct, creates a 1x1000000-vector.
If you are in doubt, use, as always, the help function! Multidimensional arrays lend themselves
for example to save the color planes of a digital image. A matrix for that purpose would be
created by a = zeros([1024 768 3]);.
As the last command in this series we will cover meshgrid.meshgrid takes two vectors as
arguments and duplicate the rst vector in the rows of the output matrix. The second vector
is duplicated along the columns of the output matrix. The number of duplications of a vector
is determined by the number of elements of the other vector. Example:
[x matrix, y matrix] = meshgrid(-10:10, -10:10); creates two matrices, whose en-
tries of the columns resp. of the rows range from 10 to 10. This command is suitable to
dene coordinate axes for two-dimensional plots. An example can be found in the chapter
Graphics in Matlab.
When working with initializations and arrays, it is important to be aware of how the contents
of these arrays are administrated in the computer. Matlab hides many tasks form our eyes,
that have to be attended to in many other programming languages rst before being able to
work with variables and arrays.
One of these tasks is the declaration of variables. In a proper programming language, one
has to determine the type of a variable when introducing it. Dierent types are oating-
point numbers, integers, strings, etc. If a numerical value is assigned to a variable, Matlab
automatically assumes that the type of the variable is intended to be a double-precision oating-
point number. Other types have to be created by an explicit type conversion.
Another task is the allocation and deallocation) of memory: An array needs memory to be
represented in the computer. Before the variable can be used, the computer has to be advised
rst to reserve a tting space in the memory for this variable (just as a real estate has to be
bought rst before one is allowed to start building a house on it). In the same manner, memory
has to be freed again if it is no longer needed (to prevent that empty real estates are left in the
memory that block the space from other house builders). Matlab hides these processes and
does all the administrative work for us. This can become critical if one writes into a variable
that was previously undened. For example the command clear; a(100000)=pi; creates a
vector with 100000 entries, and only in the 100000-th entry the number is written. If the
range of the vector, for which only 100000 elements were reserved, is now exceeded by writing
a(100001)=pi;, the following will happen:
a new vector with 100001 elements is created.
into element 100001, 2 is written.
the elements 1 bis 100000 are copied from the old to the new vector.
the memory allocated by the old vector is deallocated.
the new vector takes the name of the old vector.
...a lot of eort to append an element. This is why you should always keep in mind to properly
43
dene and initialize arrays (with zeros or ones) before using them!
6.3. Simple, predened Vector Operations
Matlab is capable to process all elements of a vector or a matrix in one go:
a = rand([1 17]);
b = rand([1 17]);
c = a+b;
The mathematical equivalent is c
i
= a
i
+ b
i
, i = 1, . . . , 17.
Vectors and matrices have to match in their dimensions; A prerequisite for the Addition of
arrays is that all dimensions have the same size. The same holds for subtraction.
Element-wise multiplication, division and exponentiation is indicated by a preceded dot:
c mult = a.*b;
c div = a./b;
This convention was introduced in Matlab to tell apart the element-wise multiplication from
the matrix multiplication.
C = [1,1;0,1]; denes the 2 2 matrix C =
_
1 1
0 1
_
D = [1,2;3,1]; denes the 2 2 matrix D =
_
1 2
3 1
_
G = C*D; gives G =
_
4 3
3 1
_
.
The mathematical equivalent is G
kl
=

j
C
kj
D
jl
.
Transposing a matrix C
t
kl
= C
lk
is done by
K = C; gives K =
_
1 0
1 1
_
The function transpose(C) can be utilized alternatively.
Inverting a matrix is accomplished by
M = D-1; gives M =
_
0.2 0.4
0.6 0.2
_
44
resp. M = inv(D);. Assuming a system of equations Dx =

b and the vector

b with

b =
_
3
1
_
is given, the solution, by using the inverse, is found in Matlab by:
_
x
y
_
= M
_
3
1
_
=
_
0.2 3 + 0.4 1
0.6 3 0.2 1
_
=
_
0.2
1.6
_
.
The solution of a system of equations can also be found by using the non-inverted matrix and
one of the symbols /, \ resp. the commands mldivide and mrdivide.
Important in a physical or geometrical context is the cross product of two vectors, which is
realized in Matlab by the command cross:
a = [0; 0; 1];
b = [1; 0; 0];
c = cross(a, b);
c
The output of these lines is:
c =
0
1
0
6.4. Reshaping, Copying and Duplication of
Matrices/Arrays
A useful command to reorganize data is reshape. The content of the array is not altered by
this command, only the organization in columns and rows, respectively the other dimensions.
The syntax is result = reshape(array, [dim1 dim2 dim3 ... dimN]);. The result
is an array of dimensions dim1 to dimN. The product of these dimension has to equal the
product of the old dimension of the array.
In the following situations, the command is sensibly applied:
Reshaping of arrays to a form that is suitable for vector operations. Let us assume that
we have 8 vectors of 10 constellations each in a 3x8x10-array wit the name r. Now we
want to rotate the vectors with a rotation matrix D ,to simulate the movement of the
night sky:
dummy = D*reshape(r, [3 80]);
r = reshape(dummy, [3 8 10]);
We rst reshaped the 8x10-matrix of vectors to form a 800-element vector, then we
applied the rotation and nally brought the result matrix back to its original form.
45
Interpretation of data, that was initially written from a le into a one-dimensional array.
Example:
f = fopen(digitalfphoto.bin, r);
a = fread(f, uint8=>uint8);
fclose(f);
imagesc(reshape(a, [500 375 3]));
set(gca, DataAspectRatio, [1 1 1]);
The second to last command reshaped the array to enable the usage of imagesc to
display the image correctly.
In the last example, the Matlab user might be annoyed that the imagesc-command projects
the rst dimension of the array (with length 500, thus e.g. the x-axis of a photo in landscape
format!) onto the y-axis. The versatile command permute remedies this situation this
command swaps single dimensions of an array. It is important to understand that not only the
organization of an array in rows and columns is changed, but also the sequence of the entries in
the memory! The syntax of the command is permuted = permute(array, [dimidx dim1,
dimidx dim2, ..., dimidx dimN]);. Here, the second argument holds the numbers from
1 to N (where N is the number of dimensions of an array, i.e. N = 2 for a matrix), that specify
the permutation of the dimensions. In the array permuted, the rst dimension will be the
dimidx dim1-th dimension of the old array array, the second dimension the dimidx dim2-th
dimension of the old array, etc.
In order to display the digital photo from our example correctly, we thus have to type:
a permuted = permute(reshape(a, [500 375 3]), [2 1 3]);
imagesc(a permuted);
This interchanges the x- and the y-direction, while the color information is preserved in the
third dimension.
repmat, another important command, duplicates an array or matrix along one or several
dimensions. The syntax is similar to reshape,
result = repmat(array, [rep dim1 rep dim2 rep dim3 ... rep dimN]);
The matrix array is cloned rep dim1-times along the rst dimension, rep dim2-times along
the second dimension, etc.
46
For example, the following commands,
a = [1 2; 3 4];
repmat(a, [3 4]) ,
give the output:
ans =
1 2 1 2 1 2 1 2
3 4 3 4 3 4 3 4
1 2 1 2 1 2 1 2
3 4 3 4 3 4 3 4
1 2 1 2 1 2 1 2
3 4 3 4 3 4 3 4
Using this command makes sense, if matrix operations are repeated with dierent operands
on the same initial vector. An example may be found in the section vectorization.
6.4.1. What Size does an Array have?
Especially when manipulating a lot of dierent arrays, when a function is supposed to operate
on an array of previously unknown size or when one wants to nd out whether two arrays are
compatible to serve as an input to a matrix operation, the following commands will be helpful:
size, length and numel.
It follows a short explanation for their function:
size: returns the size of an array in a vector that has as many elements as the array
has dimensions. Example:
size(zeros([17 42]))
ans =
17 42
length: returns the longest dimension of an array, irrespective of which dimension this
is. Example:
length(zeros([17 42]))
ans =
42
length(zeros([42 17]))
ans =
42
numel: returns the total number of all elements of an array and is identical with the
construct prod(size(array));. Example:
numel(zeros([17 42]))
ans =
47
714
6.5. Example: Vectorization of a Program
To close this chapter, we will discuss two examples of what to do when vectorizing a program.
First, consider the problem of generating a NxN-Matrix with entries a
ij
= i j. The tedious
way via loops looks like this:
N = 17;
a1 = zeros([N N]);
for i=1:N, for j=1:N; a1(i, j) = i*j; end; end
It is much simpler to rst create a row vector of the numbers 1 bis N, and subsequently
duplicate this this vector via repmat to obtain a NxN-matrix. Finally, this matrix is multiplied
element-wise with its transpose:
N = 17;
onefold = 1:N;
manifold = repmat(onefold, [N 1]);
a2 = manifold.*manifold;
However, this is still not yet the pinnacle of elegance that can be reached. Thinking a little
about this problem, one might come to the conclusion that the mathematical operation that
is intended can be described as a matrix multiplication of a Nx1-matrix with a 1xN-matrix
(the so-called outer product of two vectors):
N = 17;
onefold = 1:N;
a3 = onefold*onefold;
As a second example we consider the following program: It denes two vectors with x- and
y-coordinates, performs a translation of these coordinates and nally a rotation.
48
%%% Example: before vectorizing
x = [ 0 -1 0 -2 0 -3 0 -4 0 -5 0 0 ...
0 5 0 4 0 3 0 2 0 1 0];
y = [ 6 4 4 2 2 0 0 -2 -2 -4 -4 -6 ...
-4 -4 -2 -2 0 0 2 2 4 4 6];
%%% Translation in x- and y-direction
trans_x = 4;
trans_y = -4;
%%% Rotation
phi = pi/4;
n = numel(x);
for i=1:n
%%% apply translation to x,y
x(i) = x(i)+ trans_x;
y(i) = y(i)+ trans_y;
end
for i=1:n
%%% apply rotation to x,y
x_temp = x(i)*cos(phi)+y(i)*sin(phi);
y_temp = -x(i)*sin(phi)+y(i)*cos(phi);
x(i) = x_temp;
y(i) = y_temp;
end
As a rst step, we can subsume the 23 coordinate-tuple to a2x23-matrix, that we call r:
r = [x; y];
Here we used the semicolon to stack the row vectors on top of each other. Every column of
the matrix r now holds a coordinate tuple. In the next step, we rewrite the translation vector
as well as a column vector:
trans = [trans x; trans y];
Now we might be tempted to replace the rst for-loop by an element-wise addition of trans
and r. But beware! The dimensions of both variables are not compatible, trans has only one
column. Thus we use the command repmat, to duplicate trans 23 times. After this we can
perform the addition:
r = r+repmat(trans, [1 size(trans, 2)]);
49
Here, we used the command size to avoid having to use the constant 23 the code will thus
also work with other, longer or shorter variables x or y.
Next up is the rotation. The rotation matrix D is quickly dened,
D = [cos(phi) sin(phi); -sin(phi) cos(phi)];,
and equally quickly applied to r:
r = D*r;
In the end, we assign the result again to the variables x and y . For this we use the colon-
operator that selects whole rows of the matrix r:
x = r(1, :);
y = r(2, :);
We can even integrate rotation and translation:
r = D*(r+repmat(trans, [1 size(r, 2)]));
This gives the vectorized program code:
%%% Example: AFTER vectorization
x = [ 0 -1 0 -2 0 -3 0 -4 0 -5 0 0 ...
0 5 0 4 0 3 0 2 0 1 0];
y = [ 6 4 4 2 2 0 0 -2 -2 -4 -4 -6 ...
-4 -4 -2 -2 0 0 2 2 4 4 6];
%%% Translation in x- and y-direction
trans_x = 4;
trans_y = -4;
%%% Rotation
phi = pi/4;
%%% Definition of the vectors
r = [x; y];
trans = [trans_x; trans_y ];
D = [cos(phi) sin(phi); -sin(phi) cos(phi)];
%%% Two to three loops vectorized:
r = D*(r+repmat(trans , [1 size(r, 2)]));
%%% assign the result
x = r(1, :);
y = r(2, :);
50
Part II.
Numerical Methods in Matlab
51
7. Integration and Dierentiation
7.1. Numerical Integration
In many cases, the antidervative can not be expressed in closed form via standard functions.
This makes numerical methods for the approximation of the antiderivative necessary (quadra-
ture). The intention of this chapter is to introduce and explain the basic concepts of numerical
integration. For this, we consider denite integrals of the form
I =
_
b
a
f(x)dx
with < a < b < and suciently smooth function f(x). Such an integral can be
interpreted as the area (with sign) under the curve f(x) (see g. 7.1).
x
y
f(x)
a b
Figure 7.1.: Denite integral with boundaries a and b as the area under a curve f(x).
An example for which numerical integration is useful are one-dimensional ordinary dierential
equations, that can be written as denite integrals:
dy
dx
= f(x) and y(a) = 0 y(b) =
_
b
a
f(x)dx = I .
7.1.1. Midpoint Rule
The simplest way of numerical integration is via the midpoint rule, or also rectangle method.
Here, the interval of integration is split into sub-intervals as shows in g. 7.2. The sub-intervals
have a width of x = (ba)/N, the midpoints of these sub-intervals are x
i
= a+(i1/2)x
with i = 1, . . . , N. The points x
i
are also called supporting points.
53
x
y
a b x x x
1 2 3
x
Figure 7.2.: Scheme of the midpoint rule.
The integral is then replaced by the Riemann sum:
I I
M
=
N

i=1
f(x
i
)x
To illustrate the accuracy of this approximation, consider the following integral of the Gaussian
bell-curve
I =
_
1
0
2

e
x
2
dx.
Due to the regular occurrence of this integral in many contexts, the corresponding primitive,
the so called error-function erf(x), was dened. In the literature, we nd
I = erf(1) = 0.842701...
The following table shows the comparison of the exact solution with the result of the midpoint
rule for dierent numbers of supporting points N.
N I
M
I = I I
M
5 0.844088 -0.00138691
10 0.843047 -0.00034613
20 0.842787 -0.00008649
60 0.842711 -9.61 10
6
From the table we can derive that the error is proportional to 1/N
2
respectively x
2
. This
can also be shown analytically. For this we rst examine the part integral
I
i
=
_
x
i
+x/2
x
i
x/2
f(x)dx.
The sub-interval [x
i
x/2, x
i
+x/2] should be chosen small enough, such that the function
f(x) varies suciently little in the interval. Within this interval, we can limit the consideration
to the rst terms of the Taylor series:
f(x) = f(x
i
) + f

(x
i
)(x x
i
) + f

(x
i
)
(x x
i
)
2
2
+ f

(x
i
)
(x x
i
)
3
6
+O((x x
i
)
4
) .
54
Substitution shows that some terms vanish
I
i
= f(x
i
)x + f

(x
i
)
(x x
i
)
2
2

x
i
+x/2
x
i
x/2
+ f

(x
i
)
(x x
i
)
3
6

x
i
+x/2
x
i
x/2
+f

(x
i
)
(x x
i
)
4
24

x
i
+x/2
x
i
x/2
+O(x
5
)
= f(x
i
)x + f

(x
i
)
x
3
24
+O(x
5
) .
Writing now with central derivation
f

(x
i
) =
f

(x
i
+ x/2) f

(x
i
x/2)
x
+O(x
2
) .
Insertion yields
I
i
= f(x
i
)x + [f

(x
i
+ x/2) f

(x
i
x/2)]
x
2
24
+O(x
5
) .
Taking the sum I =

N
i=1
I
i
, many terms will vanish, thus
I I
M
=
x
2
24
[f

(b) f

(a)] +O(x
4
) .
Hence, the error is proportional to x
2
and depends on the derivations at the boundaries of
the interval.
7.1.2. Trapezoidal Rule
Instead of approximating the area by little rectangles, here, trapezoids are used (see g. 7.3).
x
2
x = a
1
x = b
4
x
3
x
y
x
Figure 7.3.: Scheme of the trapezoid rule. The dashed line illustrates the division of a trapezoid
into a triangle and a rectangle.
The width of the sub-intervals is x = (b a)/(N 1). The supporting points lie at the
borders of the sub-intervals: x
i
= a + (i 1)x with i = 1, . . . , N.
55
In the part integral I
1
, the area of the trapezoid can be split up into an area of a triangle and
the area of a rectangle (see g. 7.3),
I
1
=
_
x
2
x
1
f(x)dx f(x
1
)x +
1
2
[f(x
2
) f(x
1
)]x =
1
2
[f(x
1
) + f(x
2
)] x
It follows the compound trapezoid rule:
I I
T
=
1
2
[f(x
1
) + f(x
N
)] x +
N1

i=2
f(x
i
)x.
Note that the trapezoid rule contains the boundaries a and b of the interval. This is referred
to as a closed formula. In contrast, the midpoint rule does not contain the boundaries of the
interval (open formula). The latter is an advantage, in case that integrable singularities exist
at x = a or x = b.
The error approximation gives, in analogy to the procedure used for the midpoint rule,
I I
T
=
x
2
12
[f

(b) f

(a)] +O(x
4
) .
Thus, the trapezoid rule is less accurate by a factor of two than the midpoint rule. This is
a surprising result, considering the two gures 7.2 and 7.3 which suggest the opposite. The
seeming contradiction is resolved when realizing that the procedure of the midpoint rule partly
compensates positive and negative deviations from the exact solution.
7.1.3. Simpsons Rule
Simpsons rule is often used since it is more accurate than the midpoint or the trapezoid rule.
The idea behind Simpsons rule is to approximate the integrand function as a parabola between
three supporting points. For this parabola, the integral can be found analytically.
As in the trapezoid rule, the supporting points x
i
= a+(i 1)x are used with i = 1, . . . , N
and N odd. The width of the sub-intervals is again x = (b a)/(N 1). The compound
Simpsons rule reads
I
S
=
x
3
(f(x
1
) + 4f(x
2
) + 2f(x
3
) + . . . + 2f(x
N2
) + 4f(x
N1
) + f(x
N
)) .
The error shall be given without strict derivation:
I I
S
= O(x
4
) .
56
The error term is of higher order as compared to the trapezoid rule. Thus we expect the
error to be much smaller with the Simpsons rule. Figure. 7.4 conrms this expectation. With
the same number of supporting points, Simpsons rule gives far more accurate values for the
integral than the trapezoid rule.
Figure 7.4.: Comparison of the absolute errors between the trapezoid rule and Simpsons rule
when evaluating the integral I =
_
1
0
2

e
x
2
dx as a function of the number of
supporting points N. Note the double logarithmic plotting. For large N, rounding
errors dominate.
7.1.4. Integration with Matlab
Matlab provides several routines for integration. The most important one for one-dimensional
integrals is
y = quad(f,a,b);
Here, f is the integrand function and a and b are the boundaries of the interval. This routines
uses an adaptive Simpson rule. It is adaptive in the sense that more supporting points are
used where the function varies strongly. Invoking the routine can be done by
y = quad(@(x)exp(x),0,1);
or
y = quad(@(x)1.0./(1.0+x.2),0,1);
57
7.1.5. Miscellaneous
Further interesting methods and aspects of numerical integration are
Non-uniformly distributed supporting points:
adaptive quadrature methods
Gaussian quadrature
Improper integrals, i.e. the function or the interval of integration is boundless
Multidimensional integrals, e.g.
_ _
f(x, y) dxdy
7.2. Numerical Dierentiation
Many applications demand the approximate dierentiation of functions via a numerical method.
Finding the analytical dierentiation is for example impossible if the function to be dierenti-
ated is by itself only given numerically.
7.2.1. Right-sided Dierentiation
The dierentiation of a function f(x) is dened by
f

(x) = lim
h0
f(x + h) f(x)
h
.
Since h = 0 is numerically impossible, one has to choose h suitably small. Geometrically this
means that the sought tangent on f at the position x is approximated by the secant through
x and x + h, see g. 7.5(a).
To estimate the resulting approximation error, we consider the Taylor series up to the second
order
f(x + h) = f(x) + f

(x)h +
f

(x)
2
h
2
+O(h
3
) .
Figure 7.6 illustrates the Taylor series of the sine function at x = 0, sin (h) = h h
3
/6 +
h
5
/120 + O(h
7
). It can be seen that the Taylor series becomes a better and better approx-
imation of the sine function with increasing order. Close to the point of construction of the
Taylor series (here x = 0), the original function can be replaced in a controlled manner by its
Taylor series of a suitable order.
Rewriting the above Taylor series results in the relation
f

(x) =
f(x + h) f(x)
h
+O(h) .
58
(b)
x+h xh x
f(x+h)
(a)
x x+h
f(x)
f(x+h)
f(xh)
Figure 7.5.: Approximation of the slope of the tangent on the curve f at the position x by the
slope of the secant (a) in x and x + h resp. (b) in x h and x + h. The latter
is a decidedly better approximation.
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
1.2
h
Figure 7.6.: Sine function (continuous line) and the corresponding Taylor series including the
1. order(+), 3. order (o) and 5. order (x).
The left hand side is the exact derivative. The right hand side is the right-handed derivative
and the approximation error, also called truncation error or discretization error, which is linear
in h. The truncation error can thus be made small when choosing h accordingly. Unfortunately,
when h gets to small, rounding errors start to contribute signicantly such that an optimization
of the trade-o between truncation error and rounding error has to be found. This is illustrated
in gure 7.7 for the example of f(x) = x
2
at the position x = 1. The total error is plotted on
a logarithmic scale
(h) =

(x)
f(x + h) f(x)
h

,
as a function of h = 10
i
, i = 1, 2, . . . , 25. For f

(x) the known analytical solution 2x is

59
inserted. The linear decrease of the truncation error for h > 10
8
is apparent. For h
2 10
16
, the rounding error dominates, where is the machine accuracy. The optimal
value h 10
8
is explained by the fact that the truncation error O(h) and the rounding error
O(/h) (denominator/numerator) are in balance if h

10
8
.
Figure 7.7.: Total error of the right-sided derivation of f(x) = x
2
at x = 1. Note the double
logarithmic plotting.
7.2.2. Centered Dierentiation
Consider the following Taylor series
f(x + h) = f(x) + f

(x)h +
f

(x)
2
h
2
+
f

(x)
6
h
3
+O(h
4
)
f(x h) = f(x) f

(x)h +
f

(x)
2
h
2

(x)
6
h
3
+O(h
4
) .
Subtraction of both equation gives
f

(x) =
f(x + h) f(x h)
2h
+O(h
2
) .
The right hand side is the centered rst derivation approximation. Note that the truncation
error now decreases quadratically with the decrease of the step width h, although the evaluation
of this formula is not more extensive than the one of the right-sided derivation. In both cases,
the function f has to be calculated at two points. In contrast to the right-sided derivation,
the sought slope of the tangent on f at the position x is now approximated by the slope of
the secant through x h and x + h (see g. 7.5(b)).
60
If we add the Taylor series for f(x + h) and f(x h) instead of subtracting them, and then
divide by h
2
, we obtain the centered second derivation approximation
f

(x) =
f(x + h) 2f(x) + f(x h)
h
2
+O(h
2
) .
The truncation error is also quadratic in h.
61
8. Dierential Equations
8.1. Ordinary Dierential Equations
In many elds of science, problems occur where one or several quantities vary dependent on
time, e.g. oscillations or decay processes. The description of these processes can be formalized
by dierential equations. As a simple example, radioactive decay may serve. An amount of
radioactive material is give at time t = 0. We are looking for a function x(t) that gives the
amount of the material x which is still present at time t. The rate at time t by which the
radioactive material decays is proportional to the amount of the material x(t) that exists at
time t. From these considerations, the following dierential equation can be derived:
x = x
with decay constant > 0. This dierential equation is referred to as ordinary, since it
only includes the derivative with respect to one quantity (here time t). It is furthermore a
dierential equation of 1. order, since only the rst derivative is taken.
The solution of a dierential equation is not a number, but a function. Here, the solution
is x(t) = x
0
e
t
, which can be proven by taking the rst derivative. Unfortunately, in most
practical cases, a dierential equation can not be be solved analytically. This holds especially
for ordinary systems of dierential equations. An example is the Newtonian mechanics of a
particle
m

r =

F(r,

r, t)
with place r = (x, y, z), velocity v =

r, acceleration

r, mass m, force

F and time t. This is an
ordinary system of dierential equations of order 2, since the second derivative with respect to
time is taken. Typically one is interested in an initial value problem: Given shall be the place
r(0) and the velocity v(0) at the starting time, sought is the place r(t) and velocity v(t) at a
later time t.
Alternatively to the previous notation, the dierential equation can also be interpreted as an
ordinary system of dierential equations of 1. order with double the number of variables

r = v
m

v =

F(r, v, t) .
Using the phase space variable x = (r, v), it can be transformed to a more compact form

x =

F(x, t) .
63
Here, the rst three components of

F are equal to v and the last three components equal to

F/m. One can see that it is sucient to consider ordinary systems of dierential equations
of 1. order. Note that in the following notation, the tildeis suppressed.
8.1.1. Euler Method
Figure 8.1.: Flow plan for the Euler integration scheme.
The aim is to nd the solution of the initial value problem of the dierential equation system
(DES)

x =

F(x, t)
numerically. For this, we rst replace the dierential quotient by the right sided derivation
x(t + ) x(t)

+O() =

F(x, t)
64
respectively
x(t + ) = x(t) +

F(x, t) +O(
2
) .
The truncation error is of 2. order in the step width . By following notation we indicate the
discretization in time: t
n
= (n 1), x
n
= x(t
n
) and

F
n
=

F(x(t
n
), t
n
), where n = 1, 2, . . ..
The method of Euler nally follows:
x
n+1
= x
n
+

F
n
.
This method is illustrated in gure 8.2.
Figure 8.2.: This gure shows a comparison between the true solution (continuous line) and a
numerical solution found by the Euler method in one time step (dotted).
Schematically, the algorithm is as follows:
Choose a step width
Specify an initial condition x
1
Set the counter n to 1
While t
n
t
max
:
Evaluate the right side of the DES

F
n
Find the new places and velocities via x
n+1
= x
n
+

F
n

Increase the counter by 1

The result is a trajectory x
1
, x
2
, ..., which approximates the (in the general case unknown)
true solution x(t).
The local error is characterized as the truncation error per time step. It is of order O(
n
),
which gives n = 2 for the method of Euler. The global error is calculated from the time
interval of the length needed to calculate the trajectory t
max
= N. N is the number of time
steps. An approximation for the maximal error is
global error N (local error)
= NO(
n
) = t
max
O(
n1
) .
The global error of the method of Euler is thus of order O().
65
Example: The Planar Pendulum
mg
L
x

y
Figure 8.3.: Schematic depiction of a planar pendulum.
As an example we will discuss the planar pendulum. The pendulum consists of a point-shaped
mass m, that is xed on a massless rod of length L (see g.8.3). The only degree of freedom
of this system is the angle of deection (t) [, ]. The total energy is the sum of kinetic
and potential energy
E =
1
2
m
_
x
2
+ y
2
_
+ mgy
with gravitational acceleration g. From g. 8.3, the following relations can be extracted:
x = Lsin
y = Lcos .
Taking the time derivation while considering

L = 0
x = L

cos
y = L

sin .
This yields
x
2
+ y
2
= L
2

2
_
sin
2
+ cos
2
_
= L
2

2
.
Insertion into the energy equation nally yields
E =
1
2
mL
2

2
mgLcos .
This is the energy of the pendulum, expressed by the angle of deection and the respective
angular velocity

. An equation of motion for the angle can be derived from the conservation
of energy

E = mL
2

+ mgL

sin
!
= 0 .
This gives the equation of motion for the planar pendulum

=
g
L
sin .
66
In the approximation of small deections, the problem can be solved analytically. With sin
, the dierential equation of the harmonic oscillator follows:

=
g
L
.
We thus choose the ansatz
(t) = C
1
cos (t + C
2
)
with amplitude C
1
, phase C
2
and angular frequency . Taking the second derivative of time
yields

=
2
=
_
g
L
.
The known result is, that in the approximation of small deections, the period of oscillation
is independent of the deection.
The general case of arbitrarily big deections is not as easy to handle. We will solve this
problem numerically. The analysis above shows, that it is sensible to choose
_
L/g as the
natural unit of time, i.e.
t =

t

L
g
.
The important point is,

t is a dimensionless quantity. This choice of scaling yields
d
2

dt
2
=
d
2

t
2
g
L
=
g
L
sin .
The equation of motion can thus be written with the scaled time as

= sin .
This equation of motion is free from both units and parameters! Introducing the natural unit
thus not only prevents range errors, but also simplies the problem.
With the generalized speed =

, the equation of motion can be written as a DES of 1.
order

=
= sin .
The following is a program that solves the equations of motion numerically via the method of
Euler.

n+1
=
n
+
n

n+1
=
n
sin
n
For the sake of clarity, the program is divided into sub-programs.
Script pendulum.m:
67
%pendulum - calculating the dynamics of the pendulum via the method of Euler
clear all;
% set initial conditions and constants
theta0 = input(initialangle=);
x = [theta0*pi /180; 0];
t = 0;
tmax = input(time=);
tau = input(stepwidth=);
% count iterations
i = 1;
while (t <= tmax)
% remember angle and time
t_plot(i) = t;
th_plot(i) = x(1)*180/ pi;
x = euler_integration(t, x, tau , derive_pendulum);
t = t+tau;
i = i+1;
end
% figure
plot(t_plot , th_plot , +);
xlabel(time); ylabel(theta);
function le euler integration.m:
function xout = euler_integration(t, x, tau, derivs)
xout = x+tau*feval(derivs , t, x);
return;
function le derive pendulum.m:
function rhs = derive_pendulum(t, s)
rhs = [s(2); -sin(s(1))];
return;
All three sub-program can be written in one le. However, for this, the main program has to
be also dened as a function.
Fig. 8.4(a) shows a solution of the equation of motion by the method of Euler with step width
68
= 0.1. We observe that the amplitude, and hence the energy, of the oscillation increases
strongly with time. This violates energy conservation and has to be considered an artifact
of the discretization of time. The non-physical increase of energy can be slowed down by
choosing a smaller step width. This is demonstrated in g. 8.4(b) for the case of = 0.05.
Unfortunately, the deviation from the true solution is still considerable, despite the small step
width.
Figure 8.4.: Numerical solution of the pendulum dynamics following the method of Euler. The
initial angle is 10

, and the initial speed is 0. The step width is (a) = 0.1 and
(b) = 0.05. Note the dierent scaling of the y axes.
8.1.2. Runge-Kutta Method
Now we will discuss a method that is superior to that of Euler, since the error term is of higher
order. The idea is to nd the changes in time (derivatives) of the variable at a sampling point
that lies at the half of the interval, over which the integration step is supposed to run. The
actual integration step is then performed with the derivative at this point, and is one order
more precise than the Euler method, as we will show in the following.
We again consider the DES

x =

F(x, t) .
In the following we will compare the Taylor series
x(t + ) = x(t) +

x(t) +

x(t)

2
2
+O(
3
)
with the method of Euler with half the step width
x

(t +

2
) = x(t) +

F(x, t)

2
.
It can be seen that

F
_
x

(t +

2
), t +

2
_
=

F(x, t) +

F
x

(x,t)

F(x, t)

2
+


F
t

(x,t)

2
+O(
2
)
69
=

x(t) +
_
_

F
x

(x,t)

x(t) +

F
t

(x,t)
_
_

2
+O(
2
)
=

x(t) +

x(t)

2
+ O(
2
) .
This yields the Runge-Kutta method of 2. order
x(t + ) = x(t) +

F
_
x

(t +

2
), t +

2
_
+O(
3
)
x

(t +

2
) = x(t) +

F(x, t)

2
.
The truncation error is of 3. order in , i.e. one order higher as compared to the method of
Euler. The calculation method is illustrated in g. 8.5.
x
t
x
n+1
x
n
x
*
n
t t + /2
n n+1
t
Figure 8.5.: Scheme of the Runge-Kutta method, 2. order.
To better understand this method, we investigate a very simple dierential equation
x = x.
With the initial condition x(0) = 1, the solution is
x(t) = e
t
.
Let us now consider a translation in time. We make use of the rules of powers and construct
the Taylor series up to the fth order
x(t + ) = e
t+
= e
t
e

x(t + ) = x(t)
_
1 + +

2
2
+

3
6
+

4
24
+O(
5
)
_
.
Comparing this with the result obtained by the method of Euler
x
n+1
= x
n
+ x
n
= x
n
(1 + )
70
shows that the method of Euler gives the exact solution up to the rst order. Now we look at
the Runge-Kutta method of 2. order, which is
x
n+1
= x
n
+ x

= x
n
+

2
x
n
.
Inserting the lower equation in the upper one yields
x
n+1
= x
n
+ x
n
+

2
2
x
n
x
n+1
= x
n
(1 + +

2
2
) .
Thus we see that the Runge-Kutta method of 2. order represents the exact solution up to the
2. order.
8.1.3. Runge-Kutta Method of 4. Order
In complete analogy, the Runge-Kutta method of 4. order can be derived. We will thus skip
this procedure and directly look at the result:
x(t + ) = x(t) + (

F
1
+ 2

F
2
+ 2

F
3
+

F
4
)

6
+O(
5
)

F
1
=

F(x, t)

F
2
=

F(x +

2

F
1
, t +

2
)

F
3
=

F(x +

2

F
2
, t +

2
)

F
4
=

F(x +

F
3
, t + )
The truncation error is of 5. order in the step width . Let us rst consider again the simple
dierential equation x = x. The auxiliary quantities F
i
can be found eortlessly,
F
1
= x
F
2
= x +

2
F
1
= x +

2
x
F
3
= x +

2
F
2
= x +

2
x +

2
4
x
F
4
= x + F
3
= x + x +

2
2
x +

3
4
x.
71
Inserting and rearranging yields
x(t + ) = x(t)
_
1 + +

2
2
+

3
6
+

4
24
_
.
We can see that the Runge-Kutta method of 4. order is the exact solution up to the 4. order
of the series. We thus assume that this method needs far less time steps than the method of
Euler. Fig. 8.6(a) conrms this. The result for the planar pendulum with a small deection
angle of 10

solved by the Runge-Kutta method of 4. order is plotted. The case of big

deection angles (here 170

) is shown in g. 8.6(b). From the comparison with g. 8.6(a) we

can deduct that the period of oscillation depends on the deection of the pendulum.
Figure 8.6.: Numerical solution of the dynamics of the planar pendulum via Runge-Kutta, 4.
order. the step width is = 0.2. The initial angle is (a) 10

and (b) 170

.
8.1.4. Adaptive Methods
Thinking about the previously introduced methods, two questions arise:
Given a DES, how should the step width be chosen?
Can the step width be chose dynamically, when the variables of the DES show less
or more temporal variations? Such a case is given for example for the pendulum with a
large deection angle near 180

(see the following gure). With a exible step width,

the calculation could be accelerated a lot.
Both questions lead toward the so called adaptive methods, where the step width is adjusted
in every time time step. The procedure is the following: Find x(t + ) rst with a big time
step . This gives a rough approximation x
g
. Then calculate the far better approximation x
k
with two successive small time steps /2.
With these two approximations, a relative error can be approximated,
=
|x
k
x
g
|
|x
k
| +
,
72
Runge-Kutta method of 4. order: pendulum with initial angle = 179.5

and = 0.2.
with the machine precision . We want that this error is approximately equal to a given value

0
. To achieve this, we recall that the local error of the Runge-Kutta method of 4. order is
proportional to
5
. Thus, is proportional to
5
. The step width has to be adjusted such
that
0

5
0
. It follows for the approximated step width:

0
= S
1

_
1/5
.
Here, S
1
< 1 is a security factor, which assures that the approximated step width is not to
big. A second security factor S
2
> 1 is introduced to prevent too large steps
new
,

new
=
_

0
if /S
2
<
0
< S
2

S
2
if
0
> S
2

/S
2
if
0
< /S
2
Feasible values are S
1
= 0.9 and S
2
= 1.25. By the way: As the new initial condition for the
next time step x
k
should be used, since this value is more accurate than x
g
.
8.1.5. Matlab Solver
Matlab provides several solvers for DES. In the simplest case, these are invoked by
[t,y] = solver(function, time interval, initial values);
The objects in this command are:
73
t a column vector of length n (adaptively found by the solver) holding the times of the
output points.
y a n m matrix with the m coordinates of the solution vector y(:,i), i = 1, . . . , m.
solver, z.B.
ode23 Runge-Kutta Method (2,3) order. The abbreviation ode stems from
ordinary dierential equation.
ode45 Runge-Kutta method (4,5) order. First choose, if no information about the
properties of the system is known.
ode113 a solver of variable order. Ideal, if high accuracy is needed or if the
calculation of the right side of the DES is very intricate.
ode15s a solver of variable order for sti DES. DES are called sti when the indi-
vidual components vary on highly dierent time scales. The terminology probably
originates from the investigation of spring pendula with very sti springs, i.e. high
spring constants.
function, In the simplest form, function has the form function dx = function name(t,x).
Both arguments dx and x are column vectors, the time t is a scalar. The function value
dx has to give the derivative of the m dierential equations.
time interval, A row or column vector, e.g [t0,t1] or [t0:step width:t1]
initial values, A row or column vector of length m.
Calling the solver and the graphical representation of the solution can than be accomplished
for example by
[t,y] = ode45(derivspendulum,[0,30],[10/180*pi,0]); plot(t,y(:,1)/pi*180);
74
9. Data Analysis
Three aspects of data analysis shall be covered in this chapter: spectral analysis via the Fourier
transformation, generation of random numbers and working with probability distributions, and
the elementary methods to t models to measured data.
9.1. Fourier Analysis and Filter Operations
The Fourier analysis is the most important integral transformation in physics. It not only
allows to nd the frequency content of a temporally varying signal, it also comes in handy to
simplify the calculation of convolutions and thus applying lters eciently to data. In every
programming language, thus also in Matlab, the Fourier transformation is implemented as the
so called Fast Fourier Transform (FFT) to optimize calculation time and to spare the user
the eort to program it themselves.
9.1.1. Fourier Transformation
It has to be distinguished between the discrete Fourier transformation and the continuous
Fourier integral. First, we will dene the Fourier integral and then introduce the discrete
version. Although the computer, due to the discretization of any function x(t), always has to
use the discrete Fourier transformation, we will use the integral notation to better understand
some basic properties.
Fourier Integral
The Fourier integral can be dened as follows:
x() =
1
2
_
+

x(t) exp (it) dt (9.1)

Attention: in the literature sometimes a pre-factor of 1/

2 is used. The factor in our

denition is chosen such that it is compatible with the normalization of the Matlab-FFT which
is described later on. The Fourier transformation is reversible; the correspondent reverse
transformation is
x(t) =
_
+

x() exp(it) d . (9.2)

75
If x(t) is a real valued function, the transform x() is usually complex. For the existence of the
Fourier transformation it is sucient that x(t) is absolutely integrable, i.e.
_
+

|x(t)|dt < .
The Fourier transformation of a function is a representation of that function as a superposition
of harmonic oscillations with angular frequency = 2f and coecients x(). It is thus
nicely suited for frequency analysis of temporal signals. The complex coecients x contain
information about phase and amplitude A of the oscillation, and yield by using Eulers
identity:
x = ( x) + i( x) (9.3)
= Aexp(i) (9.4)
= Acos() + iAsin() (9.5)
Here, i is the imaginary unit and and denote the real respectively the imaginary part of
the complex number.
If x(t) is periodic, for example in 2, or if x(t) is only dened in the interval [0, 2], this can
be expressed as the Fourier series with coecients x
k
:
x
k
=
1
2
_
2
0
x(t) exp (ikt) dt . (9.6)
The reverse transformation is written as an innite sum:
x(t) =
+

k=
x(t) exp (ikt) . (9.7)
By rescaling of the t-axis by t

= 2t/T, a function a(t) dened on an interval [0, T] can

always be mapped on a function x(t

) in the interval [0, 2], and thus can be transformed by

equation (9.6).
For further properties of the Fourier transformation you may consult the instructive Wikipedia
article.
Discrete Fourier Transformation
In the computer, functions are dened on discrete sampling points in a nite interval. Let
us assume a function a(t) is sampled at N equidistant points t
n
= Tn/N, and the values
at these points are a
n
= a(t
n
). By the transformation t

= (2/T)t we transfer a(t) to a

function x(t

) in the interval [0, 2] and approximate the integral from equation (9.6) with the
midpoint rule (see chapter Integration an Dierentiation):
x
k
=
1
2
_
2
0
x(t

) exp (ikt

) dt

1
2
N1

n=0
a
n
exp (ik2t
n
/T) t

. (9.8)
Here, t

is given by (2/T)(T/N) = 2/N, which means that

x
k
A
k
=
1
N
N1

n=0
a
n
exp (i2nk/N) . (9.9)
76
This equation describes the discrete Fourier transformation, the implementation of which we
will discuss more extensively in the following paragraph. The corresponding reverse transfor-
mation is:
a
n
=
N1

k=0
A
k
exp (i2nk/N) . (9.10)
Fast Fourier Transform (FFT)
The discrete Fourier transformation is invoked in Matlab by the command a trans = fft(a);,
where a is a function or time series. The fast Fourier transform is an algorithm that is optimized
for a fast execution of the transformation. It is based on the idea to rst divide a function
of N sampling points into two function of N/2 sampling points, and then to compute two
Fourier transformations on the partial functions and combining the two results to the sought
transformation of the full function. This procedure can be iteratively extended: further partial
function then have N/4, N/8, . . . sampling points. The reverse transformation, the inverse
Fourier transformation, is invoked by the command ifft.
Calling help fft shows that the Matlab-implementation of the FFT does unfortunately not
divide the result by the number of sampling points, which is the number of element of a vector
a. This has to be done manually.
The performance gain of the FFT in comparison to a non-optimized, brute force algorithm is
impressive: the execution time of the FFT is already faster by 4 orders of magnitude(!) for
N = 2
16
sampling points:
n_power = 12;
t = zeros([ n_power 2]);
for power=1: n_power
fprintf(N=2^%i...\n, power);
n = 2^power;
a = rand([n 1]);
%%% Calculates FFT of signal a
tic;
a_fast = fft(a);
t(power , 1) = toc;
%%% Calculate Fourier transform by brute force
tic;
a_slow = zeros([n 1]);
a_slow (1) = sum(a);
exp_k1 = exp(-complex (0, 1)*2*pi *(0:(n-1))/n);
exp_k = 1;
for k=1:n-1
77
exp_k = exp_k.*exp_k1;
a_slow(k+1) = sum(a.*exp_k);
end
t(power , 2) = toc;
end
%%% Plot execution times and the
%%% acceleration factor
figure (1);
subplot (2, 1, 1);
plot(1:n_power , t);
legend({FFT Fourierbruteforce});
xlabel(poweroftwo); ylabel(t[s]);
subplot (2, 1, 2);
plot(1:n_power , t(:, 2)./t(:, 1));
xlabel(poweroftwo); ylabel(factor);
Application: The Power Spectrum
This paragraph deals with the question of how the FFT is applied to a signal a(t) , and the
interpretation of the result. For this we consider a sine wave with frequency f and a step
function as possible signals.
n = 100; dt = 0.01; f = 5;
a sine = 42*sin((0:(n-1))*dt*2*pi*f);
a step = (0:(n-1)) >= n/2;
The signals are sampled with a frequency of 1/dt for T = 1s, i.e. the discretization of the
functions has the step width dt. The FFT is calculated by the following:
fft a sine = 1/n*fft(a sine);
fft a step = 1/n*fft(a step);
Now we want to investigate how much energy is contained in the dierent frequency bands.
For this we calculate the so called power spectrum (f), i.e the square of the sum of oscillation
amplitudes for a specic frequency f. Here one has to be aware of two possible stumbling
blocks:
In the rst place we have to make sure, how Matlab stores the calculated coecients A
k
in the result vector: The rst entry, e.g. a sine(1) contains A
0
, i.e. the mean of the
function a(t). The k-th entry then holds the coecient A
k1
, for example a sine(17)
would be A
16
.
Which frequency does the coecient A
k
correspond to? A
k
corresponds to the harmonic
oscillation, for which exactly k periods t into the time interval of length T. The
frequency is thus f
k
= k/T.
Let us now look at the coecients A
k
and A
Nk
. From equation (9.9) we can see
78
Figure 9.1.: Power spectrum of a step function.
that exp(i2(N k)n/N)) = exp(i2kn/N) exp(i2n) = exp(i2kn/N). Hence,
A
Nk
is equivalent to the coecient A
k
with negative wave number k and thus
corresponds to the same frequency than A
k
.
If we want to know the amplitude of the oscillation with frequency f = 5Hz, we have to look
at the coecient with the wave number k = Tf
k
= 5 and k = 5. The correspondent value
(f) is found in Matlab-notation as:
phi 5Hz = (abs(fft a sine(1+5))+abs(fft a sine(n-5)))
The output 1.7640e+03 is indeed the square of the amplitude 42.
A signal with n samples can maximally represent oscillations with n/2 periods. Oscillations of
higher frequencies are reduced by sampling to oscillations of lower frequencies with negative
wave number. This insight is known as the Nyquist-Theorem. When calculating the power
spectrum, one can thus ignore all coecients from k = n/2 upwards and double all amplitudes
of the low frequency components (except for A
0
, |A
k
| = |A
k
| holds). A corresponding Matlab
command plots the power spectrum for a step function, which has, as expected, a broad band
spectrum:
plot(0:(n/2), [fft a step(1) 2*abs(fft a step(2:(n/2)+1))]);
xlabel(wave number k); ylabel(power);
79
9.1.2. The Convolution Theorem
The fast Fourier transform is not only useful to determine the frequency content of temporal
signals, but also to perform so called convolutions. A convolution of two functions f(t) and
g(t) is dened by the following equation:
h(t) =
_
+

f(t

)g(t t

) dt

(9.11)
Here, g is often referred to as convolution kernel. The procedure of a convolution can be
visualized as follows: We interpret f as a arbitrary function, that is compared to a template
function g. To account for possible shifts on the time axis, the template is shifted between
t = + and t = and the comparison is made at every position t. If both functions
match perfectly, the result h(t) is big. If the shifted g does not match with f, the result is
small.
Applications of this equation are numerous in physics, and reach from simple lter operations
to timely resolved frequency analysis (examples will be discussed later). First, we want to
understand the connection between convolutions and the Fourier transformation. As a short
notation of the application of the Fourier transform F to a function f (resp. its reverse
transformation F
1
) we introduce:

f(k) = F [f(t)]( k) (9.12)

f(t) = F
1
_

f(k)
__
t
_
(9.13)
Now we apply the Fourier transform to both the left and the right side of the Denition (9.11)
and gain after short computation,

h(k) = 2

f(k) g(k) , (9.14)
or, in short notation,

h = 2F[f]F[g]. To get the sought result h(t), we apply the inverse
Fourier transform to the equation, which results in
h = F
1
[2F[f] F[g]] . (9.15)
The advantage of equation (9.15) over (9.11) lies in the computation speed of FFT: despite
a three-fold transformation, calculating equation (9.15) is faster than computing the integral
in (9.11), since the convolution integral corresponds to an element-wise multiplication of the
Fourier coecients in the Fourier space k try to verify!
Non-periodic Functions
The discrete Fourier transformation can be applied to any arbitrary function f, periodicity is
not required. The convolution theorem, on the other hand, does strictly speaking only apply
for the Fourier integral. It thus demands either a denition of the function in [, +], or a
periodic function on a nite interval. Only in the latter case, fft/ifft can be directly used,
i.e. without post-treatment, to evaluate a convolution integral.
80
Still, the FFT can be used with non-periodic functions, provided the convolution kernel g is
suciently narrow, i.e. its absolute value almost vanishes within m << n sampling points
around t = 0. This is for example given for exponentially decaying functions or Gaussian
distributions. Unwanted boundary eects can be suppressed by simply discarding the rst and
last m elements of the result vector. Another possibility is the continuation of a function above
its borders by concatenating a vector of k zeros. The command is fft(f, n+k), where n is
the length of the vector f. The extension of f by zeros is only sensible in specic situations
(see also remarks further down), for example if signal was 0 before and after a period of data
acquisition, or when a lter is successively inserted into a beginning signal.
Application: Filtering of a Time Series
Let us consider the ltering of a temporal signal as an example for the application of the
convolution theorem. The signal is chosen to be a rectangle function, that is 1 in the rst
third of the data acquisition period, then jumps to 0 and increases to 2 in the last third. The
lter shall be an exponentially decaying function exp(t/), which realizes a low pass with
time constant . The following program generates the signal and the lter, applies the lter
operation and displays the result. Border eects caused by the assumed periodicity of the
signals are clearly discernible.
% parameter
n = 1000;
t = (0:(n-1))/n;
tau = 0.05;
% rectangle signal
f_signal = (t < 0.33)+2*(t > 0.66);
g_filter = 1/tau*exp(-t/tau);
f_filtered = real(ifft(1/n*fft(f_signal ).*fft(g_filter )));
% plot
figure (1);
subplot (3, 1, 1);
plot(t, f_signal ); xlabel(t); title(Signalf(t));
subplot (3, 1, 2);
plot(t, g_filter ); xlabel(t); title(Filterg(t));
subplot (3, 1, 3);
plot(t, f_filtered); xlabel(t); title(FilteredSignalh(t));
Equation 9.15 reveals a further, interesting possibility to apply lter operations: The term
F[g], the transform of the lter kernel, acts through the multiplication with F[f] similar
to a switch, that only allows passage of permitted frequencies and suppresses unwanted
frequencies. This properties can be used to directly design lters with the sought properties
in the frequency space. As an example, we could set all frequency components of a signal to
0 below a threshold frequency. This transformation realizes a high pass on f:
81
0 0.2 0.4 0.6 0.8 1
0
1
2
t
Signal f(t)
0 0.2 0.4 0.6 0.8 1
0
10
20
t
Filter g(t)
0 0.2 0.4 0.6 0.8 1
0
2
4
t
Filtered Signal h(t)
Figure 9.2.: Low pass ltering
% construct high pass filter
k_min = 5;
g_filter_highpass = zeros([1 n]);
g_filter_highpass(1+k_min:end -(k_min -1)) = 1;
f_filtered_highpass = real(ifft(1/n*fft(f_signal ).* g_filter_highpass));
% plot
figure (2);
plot(t, f_filtered_highpass); xlabel(t);
title(Highpass -FilteredSignalh(t));
When dening own lters in frequency space, one has to be cautious: most of these lter are
acausal and change the phase of f also in the not suppressed frequency bands.
To conclude, two technical remarks shall be made. First, the FFT is also dened for higher
dimensional functions. The Matlab-commands are (i)fft2, (i)fft3 and (i)fftn. This
is useful for example to lter digital images. Second, the FFT can be applied to a specic
dimension n of an N-dimensional array. The syntax is fft(a, [], n), where the empty
brackets indicate that a zero-padding, as described earlier, shall not be made. This form of
the FFT is worth considering, when several functions, that are stored in the columns or rows
of a matrix, shall be transformed in one go.
82
0 0.2 0.4 0.6 0.8 1
1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
1
x 10
3
t
HighpassFiltered Signal h(t)
Figure 9.3.: high pass ltering
4 2 0 2 4
1
0.5
0
0.5
1
t [s]
g
(
t
)
Figure 9.4.: Wavelet lter g(t) for the frequency analysis of non-stationary signals
Application: Examples for Convolution Operations
To end this section, some further examples for the application of the convolution theorem in
physics and data analysis shall be mentioned:
Wavelet Transformations. Fourier transformations are ill suited for signals, whose spec-
trum changes over time. This also includes signals which do contain periodic oscillations,
but where the frequency is subject to uctuations.
In wavelet transformations, a signals is convoluted with a lter g, that only contains few
oscillations of a specic period and decays to 0 outside of this range. An example is
depicted in the following picture:
83
A spectrum is obtained that not only depends on the frequency, but also from time. A
much used wavelet is the Morlet-wavelet, which is a harmonic oscillation multiplied with
a Gaussian curve.
Correlation Functions. The cross correlation C() between signals f(t) and g(t) is
dened by the following equation:
C() =
_
f(t)g(t + )dt (9.16)
Here, denotes the delay between the leading signal f and the for > 0 lagging
signal g. For f(t) = g(t), C is referred to as the auto-correlation.
Equation (9.16) is not directly a convolution, this means one has to be careful when
applying the convolution theorem. Utilizing that F[f(t)](k) =

f(k) it holds:
C() = F
1
[ 2F[f(t)](k)F[g(t)](k)]( ) (9.17)
The following examples come from the theoretical neurophysics:
Reverse Correlation. A simple model for the response properties of neurons in the
visual system assumes that the response r(t) of a neuron is constructed by a linear
superposition of a stimulus s(x, y, t) at the location (x, y) on the retina, multiplied with
a weight function w(x, y, ). is the delay between stimulus and neuronal response,
and g[ ] an additional point-non-linearity:
r(t) g
__
x
dx
_
y
dy
_

dw(x, y, )s(x, y, t )
_
(9.18)
Again, w can be interpreted as a lter. With this insight, the inner integral over is
best numerically solved via the convolution theorem.
When the stimulus s(x, y, t) and the response r(t) is known, the unknown lter w can be
determined under special conditions. This becomes specically easy, when the stimulus
is uncorrelated white noise:
w(x, y, )
_
r(t)s(x, y, t )dt (9.19)
After application of the convolution theorem, we get:
w(x, y, ) = F
1
[ 2F[r(t)](k)F[s(x, y, t)](k)]( ) (9.20)
Recurrent Networks. Neuronal networks usually have a one- or two-dimensional topol-
ogy, where a neuron at position x is coupled to a neuron at position x

by a weight
of magnitude w(x, x

). The dynamics of such a network are usually described as a

dierential equation for the neuronal activities A(x, t) in the following form:

A(x, t) = A(x, t) + g[I(x, t)] (9.21)
84
I(x, t) denotes the incoming current of the neuron at position x; this is a sum across
the weighted activities of all other neurons:
I(x, t) =
_
w(x, x

)A(x

, t)dx

(9.22)
If the weight are invariant to translation via w(x, x

) = w(x x

), the solution of this

integral is again a case for the well known convolution theorem.
9.2. Probabilities and Distribution Functions
9.2.1. Distributions
Often the problem arises to estimate an underlying distribution (x) from a sample {x
i
} of
cardinality N (i = 1, . . . , N). The simplest possibility is to create a histogram of the values
x
i
. For this, a range of the x-axis between [x
min
, x
max
] is divided into M intervals I
j
of
size x = (x
max
x
min
)/M. An approximation of is now given by nding the number of
elements x
i
that fall within a specic interval I
j
= [x
min
+ (j 1)x, x
min
+ jx[:
h
j
=
_
x
min
+jx
x
min
+(j1)x
N

i
(x x
i
)dx (9.23)
It thus holds (x) = h
j
/(Nx) for x in I
j
. For a good approximation, N >> M should be
given.
Matlab provide the function histc, that calculates distributions from samples. The syntax is:
h = histc(x samples, x min:Delta x:x max{, dim});
The vector h contains the number of samples that lie in the intervals determined by the second
argument (the intervals do not have to be equidistant!).
Histograms can also be calculated from multi-dimensional distributions of x. However, one
needs an extremely large number of samples, or the approximated distribution has to be
smoothed to obtain a suciently good estimation. A dierent problem pose distributions
that unevenly stretch out across big intervals of x here one should think about dynamically
adapting the binning width x.
9.2.2. Random Numbers
Creating random numbers is a often required function of a programming language. There are
numerous algorithms to create numbers that are distributed as random as possible, these will
however not be covered here. Instead we will concern ourselves with creating random numbers
from arbitrary given distributions.
85
Uniformly Distributed Random Numbers
For uniformly distributed random numbers, Matlab provides the command rand. It creates
random numbers in the interval [0, 1[ and can be used in dierent ways:
a = rand;
b = rand([n 1]);
c = rand(n);
d = rand(size(array));
The rst command gives exactly one random number, the second one a vector of n random
numbers. Attention: the third command directly gives a n x n matrix or random numbers.
The fourth variant gives an array of random numbers with the same size of the array array.
An example of the usage see the following paragraph.
It might be a good idea to generate the same sequence of random numbers when restarting
a program. In this way, faulty code can be tested more eectively or dierent numerical
procedures can be compared with the same realization of a random process. To accomplish
this in Matlab, the seed, i.e. the starting value of an internal random number generator, can
be set to a xed value by
rand(state, start value);
Normally Distributed Random Numbers
Normally distributed random numbers are created by the command randn. The syntax is
analogue to rand. The distribution has the mean 0 and the variance 1.
To generate other uniform or normal distributions, it is best to scale and move the numbers
generated by rand and randn. The following Matlab code for example draws samples from a
uniform distribution in the interval [2, 4[ and a normal distribution with mean 3 and standard
deviation 0.5, and plots the corresponding distributions via histc:
86
%%% number of samples
n_samples = 12345;
%%% draw random numbers and rescale
x_uniform = 2+2*rand([ n_samples 1]);
x_normal = 3+0.5* randn([ n_samples 1]);
%%% estimate distributions
x_axis = 1.5:0.05:4.5;
rho_uniform = histc(x_uniform , x_axis)/ n_samples/0.05;
rho_normal = histc(x_normal , x_axis)/ n_samples/0.05;
%%% plot exact and estimated distriutions
figure (1);
subplot (2, 1, 1);
bar(x_axis , rho_uniform);
hold on;
plot(x_axis , (x_axis >= 2).*(x_axis <= 4)/2, r-);
hold off;
xlabel(x); ylabel(\rho_{uniform }(x));
subplot (2, 1, 2);
bar(x_axis , rho_normal);
hold on;
plot(x_axis , 1/sqrt(2*pi )/0.5*exp( -0.5*( x_axis -3).^2/0.5^2) , r-);
hold off;
xlabel(x); ylabel(\rho_{normal}(x));
Random Numbers from Arbitrary Distributions
To generate random numbers from arbitrary distributions (x), we will schematically introduce
3 procedures:
The Hit-and-Miss Method. Let
max
be the maximum of the distribution . To generate
random numbers from (x), we have to make sure that the value x occurs with the relative
frequency (x)/
max
. This can be assured by a simple procedure that uses the random number
generator for uniformly distributed random numbers: we rst draw a candidate x
1
for a random
number from the area of denition of (x). Then we draw a second number x
2
from [0, 1[.
If this one is smaller than (x
1
)/
max
, the rst number is accepted, otherwise the procedure
is repeated. See also the following picture: only pairs of numbers (x
1
, x
2
) are accepted, that
fall into the area under the renormalized distribution .
The hit-and-miss method can be very time consuming. It is easier if the inverse function of
the primitive F of can be calculated. The idea is to start out with a uniformly distributed
87
1 1.5 2 2.5 3 3.5 4 4.5 5
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
x

u
n
i
f
o
r
m
(
x
)
1 1.5 2 2.5 3 3.5 4 4.5 5
0
0.2
0.4
0.6
0.8
1
x

n
o
r
m
a
l
(
x
)
Figure 9.5.: Uniform and normal distribution.
random number y and nd a transformation x = g(y) such that the probabilities
u
(y) dy and
(x) dx are equal. With
u
(y) = 1 one can derive
y = F(x) =
_
x

(x

) dx

, (9.24)
and thus x = g(y) = F
1
(y). Is F
1
known, one simply draws the uniformly distributed
random numbers y and calculates the sought random numbers x from the inverse function. If
F
1
can not be found analytically, there are two possibilities:
Interval Nesting. The primitive F can be numerically approximated as the integral of with
one of the numerical methods introduced in an earlier chapter for an arbitrary x. By nesting
of intervals, the value of x, that corresponds to a specic random number y, can now be
conned. This procedure is not particularly fast, but can yield a good precision.
Tabulation. If many random number have to be drawn, the inverse function F
1
(y) can rst
be tabulated. For this one is advised to use equidistant sampling points y
j
. In order to then
draw a random number x from (x), again rst a random number y
1
is taken from a uniform
distribution, and then the neighboring values y
j
and y
j+1
are found, for which y
j
y
1
< y
j+1
.
With another random number y
2
from a uniform distribution in the interval [0, 1[ the random
number x is:
x = y
2
(F
1
(y
j+1
) F
1
(y
j
)) + F
1
(y
j
) . (9.25)
88
0 2 4 6 8 10
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
1
x
2

(

m
a
x
)
Figure 9.6.: Hit-and-miss method: draw two uniformly distributed random numbers x
1
and
x
2
, and accept x
1
if both numbers fall into the blue area. By this, the probability
distribution marked by the red line.
9.3. Regression Analysis
Analysis of measured data is an important part in evaluating physical theories and models.
Usually, some parameters of the physical models have to be tted to the data and then
the adapted model shall be scrutinized whether it describes the data well. The mandatory
techniques of parameter tting as well as the evaluation of the goodness of a t will be
discussed in the following.
9.3.1. General Theory
Let us assume that data is provided as pairs of values (x
i
, y
i
), i = 1, . . . , N. The model
that is to be tted predicts a functional correlation of the form Y (x; a
1
, . . . , a
M
) between the
quantities x and y; a
1
, . . . , a
M
are here the parameters that have to be found. Of course there
should be much more data points than parameters, mathematically speaking: N M. One
example, that is commonly encountered in the practical courses, is the tting of a straight line
Y (x; a
1
, a
2
) = a
1
+ a
2
x
to a data set.
It is the aim to adjust the values of these parameters such that the function Y (x; a
1
, . . . , a
M
)
represents the date as well as possible. Intuitively we would assume that in case a t is good,
the graph of the function Y (x; a
1
, . . . , a
M
) will lie close to the points (x
i
, y
i
) (see gure 9.7).
We can quantify this statement by measuring the deviations between the data points and the
function

i
= Y (x
i
; a
1
, . . . , a
M
) y
i
.
89
x
1
x
2
x
3
x
4
y
4
y
3
y
1
y
2
x
y
Y(x)

i
Figure 9.7.: Fitting a curve to data points.
We choose our tting criterion such that the sum of the squares of the deviations becomes
minimal. This means that we have to tune our parameter values a
1
, . . . , a
M
such that the
cost function

2
(a
1
, . . . , a
M
) =
N

i=1

2
i
=
N

i=1
[Y (x
i
; a
1
, . . . , a
M
) y
i
]
2
is minimized. This method is dubbed least squares method. It is not the only possible method,
but the one that is most commonly used.
Usually the data points have an estimated error bar (the condence interval), that we denote
as y
i

i
. In this case we should change our tting criterion such that points with bigger
error bars have a smaller weight. Following this logic we dene

2
(a
1
, . . . , a
M
) =
N

i=1
_

i
_
2
=
N

i=1
[Y (x
i
; a
1
, . . . , a
M
) y
i
]
2

2
i
. (9.26)
9.3.2. Linear Regression
We now want to t a straight line to the data points,
Y (x; a
1
, a
2
) = a
1
+ a
2
x.
This type of tting is called linear regression. The two parameters a
1
and a
2
have to be chosen
such that

2
(a
1
, a
2
) =
N

i=1
[a
1
+ a
2
x
i
y
i
]
2

2
i
becomes minimal. The minimum can be found by taking the derivation of the equation and
setting it to zero

2
a
1
= 2
N

i=1
a
1
+ a
2
x
i
y
i

2
i
= 0

2
a
2
= 2
N

i=1
x
i
a
1
+ a
2
x
i
y
i

2
i
= 0 .
90
We now introduce the following quantities
=
N

i=1
1

2
i
,
x
=
N

i=1
x
i

2
i
,
y
=
N

i=1
y
i

2
i
,

x
2 =
N

i=1
x
2
i

2
i
,
xy
=
N

i=1
x
i
y
i

2
i
.
These sums are calculated directly from the data points, are thus known constants. Hence we
can rewrite the previous system of equations as
a
1
+ a
2

y
= 0
a
1

x
+ a
2

x
2
xy
= 0 .
This is a system of linear equations with two unknowns a
1
and a
2
. The solutions are
a
1
=

y

x
2
x

xy

x
2 (
x
)
2
, a
2
=

xy

x
2 (
x
)
2
. (9.27)
In a second step we want to estimate the error bars for the parameters a
1
and a
2
. We use the
law of error propagation

2
a
j
=
N

i=1
_
a
j
y
i
_
2

2
i
.
Insertion of the equations (9.27) yields

a
1
=

x
2

x
2 (
x
)
2
,
a
2
=

x
2 (
x
)
2
.
As an example we consider g. 9.8. Here the t parameters are a
1
= 0.1529 0.2633 and
a
2
= 1.0939 0.0670. Note that the error bars
a
j
do not depend on the y
i
. These error bars
are thus no quantier of the goodness of the t.
9.3.3. Goodness of the Fit
It is clear that the t is locally good, if the deviation of the function is smaller or approximately
equal to the error bar. We consider the upper boundary
|y
i
Y (x
i
)|
i
.
Insertion into the denition of
2
in equation (9.26) yields
2
N. The more parameters are
used, the better the t will be. For the case of N = M, the t will be exact. For the case of
the straight line, this simply means that to N = 2 points, we can always exactly t a straight
line (M = 2). Thus, a good criterion for the goodness of the t is

2
N M .
As an example we again refer to g. 9.8. Here,
2
4.5 and N M = 8. The goodness of
the t is thus quite high.
91
0 2 4 6 8 10
0
2
4
6
8
10
12
x
i
y
i
,

Y
(
x
)
Figure 9.8.: An example of linear regression. A straight line Y (x) is tted to the data points
(x
i
, y
i
) .
9.3.4. Non-linear Regression
In many cases, the tting of a non-linear function can be broken down through a clever variable
transformation to the tting of a linear function. As the rst example, the commonly occurring
case
Z(x; , ) = e
x
shall be observed. One thinks for example of exponential decays. With the variable transfor-
mation
Y = ln Z, a
1
= ln , a
2
=
we get the linear function
Y (x; a
1
, a
2
) = a
1
+ a
2
x.
A second example are power laws of the type
Z(t; , ) = t

.
the variable transformation
Y = ln Z, x = ln t, a
1
= ln , a
2
=
also gives a linear function.
92