Dynamical Systems

LECTURE 1: INTRODUCTION
1. Introduction
NONLINEAR DYNAMICS AND CHAOS
2. Maps
3. Flows
4. Fractals and Attractors
Patrick E McSharry
Systems Analysis, Modelling & Prediction Group
5. Bifurcations
6. Quantifying Chaos
www.eng.ox.ac.uk/samp
7. Nonlinear Time Series Analysis
patrick@mcsharry.net
8. Nonlinear Modelling and Forecasting
Tel: +44 20 8123 1574
9. Real-World Applications
Trinity Term 2007, Weeks 3 and 4

Mondays, Wednesdays & Fridays 09:00 - 11:00
10. Weather Forecasting

11. Biomedical Models
12. Time Series Analysis Workshop
Seminar Room 2
Mathematical Institute
University of Oxford
c 2007 Patrick McSharry p.1

Nonlinear dynamics and chaos
Suggested Reading

Relevant journal articles
Strogatz, S. H., Nonlinear Dynamics and Chaos: With Applications to Physics, Biology,
Chemistry, and Engineering, Reading, MA: Addison-Wesley (1994)
Yorke, J. and Li, T. Y., Period Three Implies Chaos, American Mathematical Monthly
82:985-992 (1975)
Eubank, S., and D. Farmer, An introduction to chaos and randomness. In Jen, E. (Ed.), 1989
Lectures in Complex Systems. Santa Fe Institute Studies in the Sciences of Complexity,
Lecture Vol. II, pp. 75-190. Reading, MA: Addison-Wesley, (1990)
May, R., Simple mathematical models with very complicated dynamics. Nature 261:
459-467 (1976)
Ott, E. and Sauer, T. and Yorke, J. Coping with Chaos, J. A. John Wiley & Sons, New York
(1984)
Ott, E., Chaos in Dynamical Systems, Cambridge: Cambridge University Press (1993)
Packard, N. and Crutchfield, J. and Farmer, J. D. and Shaw, R., Geometry from a time
series, Phys. Rev. Lett. 45: 712-716 (1980)
Crutchfield, J. P, N. H. Packard, J. D. Farmer, and R. S. Shaw. (1986) Chaos, Scientific
American 255:46-57 (1996)
Schuster, H.G., Deterministic Chaos: An Introduction, VCH, (1995).

Wiggins, S., Introduction to Applied Nonlinear Dynamical Systems and Chaos, Springer
(1990)
Kantz, H. and Schreiber, T., Nonlinear Time Series Analysis, Cambridge Univ. Press (1997)
Abarbanel, H.D.I, Analysis of Observed Chaotic Data, Springer, (1996)


Sources of information
Suggested special topics
Journals:
Physica D
Physical Review Letters
Physical Review E
Physics Letters A
International Journal of Bifurcation and Chaos
Look for evidence of low-dimensionality in real time series

Investigate the predictability of a real time series using nonlinear methods
Online discussion groups:

sci.nonlinear
www.jiscmail.ac.uk/lists/allstat.html
www.jiscmail.ac.uk/lists/timeseries.html
Websites:
www.societyforchaostheory.org (Society for chaos theory)
www.physionet.org (MIT-Harvard biomedical database and tools)
www.comdig.org (Complexity Digest)
Available time series:

electronic circuits
lasers
sunspot record
electricity data (demand, price, grid frequency)
weather data (temperature, precipitation, wind speed)
economic data (GDP, inflation, interest rates, unemployment)
financial data (stockmarket tick data, S&P500, Dow Jones, FTSE100)
biomedical signals (ECG, EEG, blood pressure, respiration, blood gases)
Construct and investigate a nonlinear model of a particular system
Compare nonlinear methods for forecasting, (e.g. RBFs vs. local linear)
Develop new methods for classification of health/disease using biomedical signals
TISEAN package
Matlab software

Special topic instructions

Linear analysis
Definition of linearity:
L(ax) = aL(x)
Should be one week of work
L(x + y) = L(x) + L(y)
Marking scheme:content 20 pts

presentation 5 pts
Principle of superposition: If x and y are solutions, then z = ax + by is also a solution

Advantages of linear models:
Often have analytical solutions
A large body of historical knowledge for helping with model specification and estimation
Less parameters - smaller chance of overfitting
Can employ Fourier spectral analysis and associated techniques
Two copies of final report required
Disadvantages of linear models:

Real-world systems are usually nonlinear
Linearity is a first order approximation and neglects higher orders
Stanislaw Ulam: nonlinear science is like non-elephant zoology
In practice, while underlying dynamics may be nonlinear, observed data may only provide
sufficient resolution for linear models
Need relevant null hypothesis tests for nonlinearity (surrogate data)


Normal distributions
Dynamical systems I
Variables
Linear
n=1
Growth, decay, equilibrium
Mean x0 and variance 2 :

(x x0 )2
exp
p(x) =
2 2
2 2
1
Easily manipulated: if x N (0, x2 ) and y N (0, y2 ), then x + y N (0, x2 + y2 )

Central limit theorem: sum of a large number of IID random variables (with finite mean and
variance) is normally distributed
For linear models:
Normal distributions are preserved by principle of superposition
Normally distributed forecast errors: Maximum likelihood gives least squares
Useful for calculating prediction intervals
Problems for nonlinear systems:
Use of normal distributions neglects possibility of asymmetric distributions
Fat tailed distributions imply larger probability of worse case scenarios (risk
management)
Exponential growth
Fixed points
RC circuit
Bifurcations
Radioactive decay
Overdamped systems, relaxational dynamics

Logistic equation for single species
n=2
All higher order moments are given in terms of x0 and
Nonlinear
Oscillations
Linear oscillator
Pendulum
Mass and spring
Anharmonic oscillator
RLC circuit
Limit cycles
2-body problem
Biological oscillators
Predator-prey cycles
Nonlinear electronics (van Der Pol)
n3
Chaos
Civil engineering
Strange attractors (Lorenz)
Electrical engineering
3-body problem (Poincare)

Chemical kinetics
Iterated maps (Feigenbaum)
Fractals (Mandelbrot)
Forced nonlinear oscillators (Levison, Smale)

Dynamical systems II
Variables
Linear
n1
Collective phenomena

Napoleans Armys Russian Campaign

Nonlinear
Coupled harmonic oscillators
Coupled nonlinear oscillators
Solid-state physics
Lasers, nonlinear optics
Molecular dynamics
Nonequilibrium statistical mechanics
Equilibrium statistical mechanics
Nonlinear solid-state physics

Heart cell synchronisation
Neural networks
Economics
Continuum
Waves and patterns
Spatio-temporal complexity
Elasticity
Nonlinear waves (shocks, solitons)
Wave equations
Plasmas
Electromagnetism (Maxwell)
Earthquakes
Quantum mechanics (Schrodinger,

Heisenberg)
General relativity (Einstein)
Heat and diffusion
Quantum field theory
Acoustics
Reaction-diffusion, biological waves
Viscous fluids
Fibrillation
Epilepsy
Turbulent fluids
Map drawn by the French engineer Charles Joseph Minard in 1861 to show the tremendous
losses of Napoleans army during his Russian Campaign of 1812
Life

Happiness time evolution
Happiness versus GDP
From Culture and Subjective Well-being, edited by Ed Diener & Eunkook M. Suh (2002)
From Culture and Subjective Well-being, edited by Ed Diener & Eunkook M. Suh (2002)
Time series from nonlinear systems

Mathematical characterisation
Time series: variables are recorded as a function of time

A
Steady state: a constant solution of a mathematical equation

e.g. Homeostasis: relative constancy of the internal environment with respect to variables
such as blood sugar, blood gases, blood pressure and pH. Control mechanisms constrain
variables to narrow limits. e.g. Following a hemorrhage, reflex mechanisms quickly restore
blood pressure to equilibrium values.
Oscillations: periodic solutions of mathematical equations

e.g. Heartbeat, respiration, sleep-wake cycles and reproduction
Irregular activity: intrinsic fluctuations, can even be present when external parameters are
relatively constant. Two distinct mathematical descriptions: noise and chaos
Noise:
Variability cannot be linked with any underlying stationary or periodic process
e.g. Fluctuating environment: eating, exercise, rest and posture affects heart rate, blood
pressure, blood-sugar levels and insulin levels
e.g. Respiratory Sinus Arrhythmia (RSA): heart rate increases during inspiration
Chaos:
Irregularity that arises in a deterministic system
Chaos can exist without influence of external noise


Noise versus Chaos
An exponential distribution, but not a Poisson process

2
Inter-event time sequences (e.g. heartbeat, neurons firing)
1.5
Believed to be a random process
ti+1
Simplest model for such a random process is a Poisson process:

Probability of event to occur in a very short time increment dt is Rdt
The probability R is independent of the previous history
Probability of two or more events occurring during dt is negligible
(Rt)k Rt
e
k!
st
1) following
(Poisson distribution)
Probability that interval between event and (k +
event is
R(Rt)k Rt
e
k!
0.5
0.1
0.2
0.3
0.4
0.5
ti
0.6
0.7
0.8
0.9
90
100
0.9
ln2/3
1.4
1.2
Probability of k events in time interval t is Pk (t) =

pk (t) =
1
ti
0.8
0.6
(PDF of Poisson process)
0.4
0.2
Average time between events is 1/R and variance is 1/R2
Nonlinear map, ti+1 = (1/R) ln |1 2 exp(Rti )| also gives p(t) = ReRt
10
20
30
40
50
i
60
70
80
0.1
0.2
0.3
0.4
0.5
t
0.6
0.7
0.8
Observation of an exponential probability density is not sufficient to identify a Poisson

process!
3.5
3
2.5
Use recurrence plots (e.g. ti+1 versus ti ) to identify structural equations
p(t)
2
1.5
1
0.5
0

Determinism and Predictability

Poincar
After Newton people believed in a deterministic, and hence, predictable Universe

Given for one instant an intelligence which could comprehend all the forces by which nature
is animated and the respective situation of the beings who compose itan intelligence and
sufficiently vast to submit these data to analysisit would embrace in the same formula the
movements of the greatest bodies of the universe and those of the lightest atom; for it,
nothing would be uncertain and the future, as the past, would be present before its eyes.
[P.S. Laplace, 1814]
Laplacian dream excludes stochastic laws of physics
Laplace acknowledged that we would never achieve the intelligence requireda tacit
appreciation that deterministic systems might not, in practice, be predictable
deterministic 6= predictable
After tackling the 3-body problem Poincar identified the phenomenon of sensitive
dependence on initial conditions (SDIC), this provided a definition of chaos
If we knew exactly the laws of nature and the situation of the universe at the initial moment,
we could predict exactly the situation of that same universe at a succeeding moment. But
even if it were the case that the natural laws had no longer any secret for us, we could still
only know the initial situation approximately. If that enabled us to predict the succeeding
situation with the same approximation, that is all we require, and we should say that the
phenomenon had been predicted, that is is governed by laws. But it is not always so; it may
happen that small differences in the initial conditions produce very great ones in the final
phenomena. A small error in the former will produce an enormous error in the latter.
Prediction becomes impossible, and we have the fortuitous phenomenon.[H. Poincar,
1903]
Laplace saw probabilities as a way to describe our ignorance of a deterministic system

Analytic expediency means most mathematics revolves around linear systems


Lorenzs butterfly effect
Sensitive dependence on initial condition
50
20
45
15
40
35
10
30
z
25
20
x(t) 0
15
10
5
10
0
20
15
15
10
0
x
10
15
20
Perfect model and perfect knowledge of observational uncertainty

Predictability varies with position
20
4
time [secs]

Chaos

Two faces of Chaos
Advent of digital computer allowed numerical investigation of nonlinear equations
The word chaos refers to disorder and extreme confusion
Lorenz found SDIC in a numerical model of the atmosphere and constructed the Lorenz
system to illustrate the effect in a simple system [1963]
To a scientist, it implies deterministic disorder
Yorke and Li coined the word chaos in 1975
On the contrary, a chaotic deterministic system is, in principle, perfectly predictable
May demonstrates chaos in the one-dimensional Logistic map in 1976
The sensitive dependence of the system dynamics to the initial conditions (SDIC) implies
that, in reality, any error in specifying the initial condition will lead to an erroneous prediction
Chaos becomes trendy, Chaos is published by James Gleick, 1987

Claims of chaos in the brain, heart, economy, stockmarket, ...
Investigations of nonlinear dynamical systems, claims of chaos are played down!
This might suggest that a chaotic system should be unpredictable
Laplace suggests using probabilistic predictions to overcome the problems, of diverging

trajectories, posed by chaotic systems
Chaos is sometimes used as a scapegoat: meteorologists blame chaos for inaccurate
predictions when it is often model inadequacy that is at fault
Important research: separating model inadequacy (structural and parametrical errors) from
effects of observational uncertainty


Deterministic versus Stochastic
Deterministic versus Stochastic
stochastic system
Deterministic:
xt+1 = axt + t
t : N (0, 1)
deterministic system
Everything is described by a single point in state space. This description

completely determines the future.
Stochastic:
Knowledge of present states does not determine the evolution of future states.
Can you tell if a system is stochastic or deterministic?
yt+1 = ayt + zt
Should a system be modelled as stochastic or deterministic?
zt+1 = 4zt (1 zt )
High dimensional deterministic chaotic system might be modelled as a stochastic system

State space trajectory of an autonomous, deterministic system never crosses itself
Sources of forecast uncertainty:
Uncertainty in the initial condition
Model inadequacy (parametrical and structural)

Introduction to Dynamical Systems

Properties of dynamical systems
A state is an array of numbers that provides sufficient information to describe the future
evolution of the system.
If m numbers are required, then these form an m-dimensional state vector x.
The collection of these state vectors defines an m-dimensional state space.
The rule for evolving from one state to another may be expressed as a discrete map or a
continuous flow:
map xt+1 = F (xt )
flow x(t)
= f (x(t))
Fixed point of a map: x0 = F (x0 )
Fixed point of a flow: x 0 = f (x0 ) = 0
Determinism: trajectories should not diverge when going forward in time

Invertibility: A dynamical system is invertible if each state x(t) has a unique predecessor
x(t 1). This implies that trajectories should never merge.
Thus continuous deterministic flows are always invertible!
Maps derived from flows (Poincar maps) are also invertible
Reversibility: if the dynamical system obtained by the transformation t t is equivalent to
the original one
Invariance under coordinate transforms: an invariant of a dynamical system represents a
fundamental property of that dynamical system, e.g. dimensions and Lyapunov exponents
System invariant offer a means of summarising the behaviour of a particular system: (e.g.
health and disease)
Non-autonomous system: x = f (x, t)

Autonomous system: x = f (x)
If the non-autonomous nature is due to periodic terms it can be made autonomous
Dissipative flow: f < 0 implies contracting state space volume
Hamiltonian systems are non-dissipative or conservative, preserving state space volume

(Liouville theorem)

Simple Pendulum
Fixed Points
An example of a simple two-dimensional dynamical system
Consider the fixed point of a flow f (x0 ) = 0
From Newtonss second law, knowledge of the forces, position and velocity are sufficient to
determine future motion
Let x(t) = x0 + (t)

= f (x0 + )
Pendulum (constrained to move in the plane)
= f (x0 ) + Dx f (x0 ) + O(||||2 )
Dynamics fully specified by the displacement angle (t) and the angular velocity (t)
Dx f (x0 ) = J
State vector given by x(t) = [(t), (t)]

Let m be the mass of the pendulum
J is Jacobian matrix of partial derivatives
g is the acceleration due to gravity
The solution is
(t) = eJt 0
l is the length of the pendulum

Let i be (distinct) eigenvalues of J
Tangential restoring force due to gravity: mg sin

Tangential force due to angular acceleration: ml
P 1 JP =
In the absence of friction, dynamics are governed by

d
dt
d
dt
where ii = i and ij = 0 if i 6= j
Let = P y so
y = et y0
g
sin
l
Classification of fixed points
<(i ) 6= 0 for all i: hyperbolic fixed point

Otherwise non-hyperbolic fixed point
Hyperbolic fixed points:
sinks: all <(i ) < 0
sources: all <(i ) > 0

saddle point: some positive and some negative
If =(i ) = 0: node
If =(i ) 6= 0: focus
Non-hyperbolic fixed points:
if <(i ) > 0 for some i: unstable
some <(i ) < 0 some = 0: neutrally stable

all <(i ) = 0: centre



Dynamical Systems

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Dynamical Systems

Transféré par

Droits d'auteur :

Formats disponibles

LECTURE 1: INTRODUCTION

NONLINEAR DYNAMICS AND CHAOS

7. Nonlinear Time Series Analysis

8. Nonlinear Modelling and Forecasting

Tel: +44 20 8123 1574

Trinity Term 2007, Weeks 3 and 4

10. Weather Forecasting

c 2007 Patrick McSharry p.1

c 2007 Patrick McSharry p.2

Relevant journal articles

Schuster, H.G., Deterministic Chaos: An Introduction, VCH, (1995).

c 2007 Patrick McSharry p.3

c 2007 Patrick McSharry p.4

Suggested special topics

Look for evidence of low-dimensionality in real time series

Online discussion groups:

Available time series:

c 2007 Patrick McSharry p.5

Special topic instructions

c 2007 Patrick McSharry p.6

Should be one week of work

L(x + y) = L(x) + L(y)

Marking scheme:content 20 pts

Principle of superposition: If x and y are solutions, then z = ax + by is also a solution

Two copies of final report required

Disadvantages of linear models:

c 2007 Patrick McSharry p.7

c 2007 Patrick McSharry p.8

Growth, decay, equilibrium

Mean x0 and variance 2 :

Easily manipulated: if x N (0, x2 ) and y N (0, y2 ), then x + y N (0, x2 + y2 )

Overdamped systems, relaxational dynamics

All higher order moments are given in terms of x0 and

Mass and spring

Strange attractors (Lorenz)

3-body problem (Poincare)

c 2007 Patrick McSharry p.9

c 2007 Patrick McSharry p.10

Napoleans Armys Russian Campaign

Coupled harmonic oscillators

Coupled nonlinear oscillators

Lasers, nonlinear optics

Nonequilibrium statistical mechanics

Equilibrium statistical mechanics

Nonlinear solid-state physics

Waves and patterns

Nonlinear waves (shocks, solitons)

Quantum mechanics (Schrodinger,

General relativity (Einstein)

Heat and diffusion

Quantum field theory

Reaction-diffusion, biological waves

c 2007 Patrick McSharry p.12

Happiness time evolution

Happiness versus GDP

Time series from nonlinear systems

c 2007 Patrick McSharry p.14

Time series: variables are recorded as a function of time

Steady state: a constant solution of a mathematical equation

Oscillations: periodic solutions of mathematical equations

c 2007 Patrick McSharry p.15

c 2007 Patrick McSharry p.16

Noise versus Chaos

An exponential distribution, but not a Poisson process