Monahan 1

Introduction
An Introduction to Probability and

Stochastic Processes for Ocean,
Atmosphere, and Climate Dynamics
1: Basic Probability
Adam Monahan
monahana@uvic.ca
Climate is what you expect, weather is what you get.

- Robert A. Heinlein
expectation lies at heart of notion of climate
this is a fundamentally probabilistic perspective
Probability is a natural way of looking at atmosphere, ocean, and
climate dynamics, both in terms of
statistics (probability and data)
stochastics (the dynamics of probability)
These lectures will provide an overview of probability theory &
stochastic processes in the context of atmosphere, ocean, and climate
dynamics
School of Earth and Ocean Sciences, University of Victoria

An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability p. 1/23
Probability: Basic concepts
Probability densities
Probabilities are used to characterise processes with indeterminate

outcomes - that is, that are random
Our main focus will be case of basic outcome space of continuous

variables = {x = R}
Any given random process X is associated with a set of possible

basic outcomes
Define differential probability of x falling in small volume dx
= {x1 , x2 , ..., xn }
(which doesnt have to be discrete set)
Can define an event A as any subset of these basic outcomes, and
assign to that a probability P (A) of its occurrence, with the rules:
(i)
(ii)
(iii)
0 P (A) 1
P () = 1 (somethings got to happen)
P (A or B) = P (A) + P (B) P (A and B)
dP (x0 ) = P (x0 x x0 + dx)

If P (x) smooth, can define probability density function (pdf)
P (x0 x x0 + dx) = px (x) dx
for which
px (x) dx = 1
Can also define cumulative distribution function

Z x0
px (x) dx
F (x0 ) = P (x x0 ) =
(additivity of mutually exclusive events)

Joint and marginal pdfs
Conditional probabilities
Probability of A and B together is joint probability

P (A andB) = P (A, B)
The conditional probability P (A|B) measures the probability of A

given that B is known to be true
Conditional probabilities are related to joint probabilities:
For x RN , px (x) = px (x1 , x2 , ..., xN ) is the joint pdf of multiple

scalar random variables
Can find pdf over subspace of x (the marginal pdf) by integrating
out other variables
E.g. in
R2 :
P (A|B) is a measure of the dependence of A and B

Independence: B irrelevant to A, so P (A|B) = P (A) and
P (A, B) = P (A)P (B)
px1 (x1 ) =
px (x1 , x2 ) dx2
px2 (x2 ) =
px (x1 , x2 ) dx1
Bayes Theorem
p(x|y) =
For x, y independent, joint pdf factors as product of marginals:

pxy (x, y) = px (x)py (y) (independence)
Expectation and moments
From properties of conditional distributions
P (A, B) = P (A|B)P (B) = P (B|A)P (A)
p(y|x)p(x)
p(y|x)p(x)
p(y|x)p(x)
=R
=R
p(y)
p(x, y)dx
p(y|x)p(x)dx
p(x|y) = R
p(y|x)p(x)
p(y|x)p(x)dx
Provides formal model of knowledge acquisition of system

variable x through experimental outcome y:
p(x) initial uncertain knowledge of x (the prior)
p(y|x) predicted outcome of experiment
p(x|y) improved knowledge of x given outcome of experiment y
(the posterior)
The pdf px (x) defines a measure of relative probabilities, leading to

the definition of the expectation (average) of a function f (x):
Z
E{f } = f (x)px (x) dx
Can interpret marginal pdf as expectation of conditional:
Z
Z
px1 (x1 ) = px (x1 , x2 ) dx2 = p(x1 |x2 )px2 (x2 ) dx2
Expectation used to define moments, which are useful
characterisations of a pdf
For scalar random variables:
mean(x) = E{x}
var(x) = std2 (x) = E{(x E{x})2 }
pdf centre
pdf spread
Expectation and moments
Moments of sea surface winds
Two common higher-order (reduced) moments:
mean(U) (m/s)
skewness
40
kurtosis
E{x})3 }
E{(x
skew(x) =
std3 (x)
12
50
10
20
E{(x E{x})4 }
kurt(x) =
3
std4 (x)
measures asymmetry of pdf
mean(w) (ms1)
10
60
0
20
50
100
150
200
250
300
10
350
50
0
std(U) (m/s)
50
100
60
measures flatness of pdf
40
60
40
150
200
std(w) (ms1)
250
300
350
50
20
skew(x) > 0
skew(x) = 0
skew(x) < 0
0.8
0.6
0.8
0.5
p (x)
0.6
20
kurt(x) > 0
kurt(x) = 0
kurt(x) < 0
0.7
p (x)
1.2
60
0
4
100
150
250
300
350
40
0
x
50
100
100
150
200
250
300
350
skew(w)
150
200
250
300
350
1
1
50
0.5
0
0
0.5
50
0
Covariance and correlation
50
2
50
0
20
60
20
50
Zonal Wind
100
150
200
250
300
350
Wind speed
Covariance and correlation
Given two variables x and y with joint pdf pxy (x, y) define
covariance
and correlation
cov(x, y)
std(x)std(y)
4
4 4
4
(c)
x2
4
4
(b)
4
4
4
These are measures of dependence between x and y

If x and y are independent, covariance and correlation vanish - but
not vice versa! (in general)
(a)
x2
cov(x, y) = E{(x E{x})(y E{y})}

corr(x, y) =
200
40
0.1
1
50
skew(U)
0.2
0.2
60
0.4
40
0.3
0.4
0
0
1.4
x1
4
4 4
(a) indep, uncorr

(b) dep, corr
(c) dep, uncorr
(d) dep, corr
(d)
x1
Families of pdfs
Covariance matrix & EOFs

For x RN define covariance matrix

xx = E (x E{x})(x E{x})T
There are many families of pdfs characterised by one or more

parameters whose properties are well-studied
The most important of these are the two-parameter Gaussian (or
normal) distributions
so var(x) = diag(xx )
Efficiently described in terms of eigenvectors ek known as empirical

orthogonal functions (EOFs):
For a Gaussian scalar x

(x )2
px (x) =
exp
2 2
2 2
1
EOFs orthogonal basis in RN which efficiently partitions variance

In EOF basis new (uncorrelated) variables given by projections
ak = x ek with diagonal covariance matrix
0.4
0.35
0.3
0.25
such that
px(x)
xx ek = k ek
mean(x) =
std(x) =
skew(x) = 0
kurt(x) = 0
0.2
0.15
0.1
0.05
(aa )ij = j ij
0
4
0
(x)/
Central limit theorem and law of large numbers
Multivariate Gaussian
One reason that Gaussian distributions are important is the central

limit theorem, which states (loosely) that if xn is a sequence of
independent mean-zero random variables with finite variance, then
Random vector X RN is multivariate Gaussian when pdf is

1
1
T 1
exp (x ) xx (x )
p(x) = p
2
(2)N det xx
where
1 X
Zn =
xi
n
= mean(x)
i=1
xx = cov(x, x)
4
3
2
For a multivariate Gaussian
all marginals are Gaussian
1 2
EOF
Another useful asymptotic result is the law of large numbers: for xn

a sequence of independent and identically distributed random
n
variables
1X
xn mean(x)
lim
n n
EOF
1
x
corr(xi , xj ) = 0
x1 , x2 independent
becomes Gaussian with finite variance as n
p(x ,x )
0
1
2
i=1
3
4
5
That is: as n becomes very large, fluctuations in sum vanish

0
x
Maximum entropy pdfs
Change of variables
Get moments from pdf; but how can we go the other way?
Suppose y = f (x) where f is invertible
The entropy
Variable y is just another way of talking about x; intuitively expect

H=
px (x) ln px (x)dx
P (y1 < y < y2 ) = P (x1 < x < x2 )
measures information content of a pdf

Given first N moments, can find pdf which maximises H subject to
the constraint it must have specified moments; takes form
if y1 = f (x1 ) and y2 = f (x2 ) conservation of probability

In terms of densities:
so
where Li are Lagrange multipliers of constrained optimisation

Provides least biased pdf with given information
Only mean and variance given, maximum entropy pdf Gaussian
py (y) =
For x, y RN
Change of variables: example
pw (w, ) = wpuv (w cos , w sin )
py (y) =
px (f 1 (y))
df /dx
px (f 1 (y))
det(fi /xj )
Case study: Tropical Pacific SST
Know the joint pdf of zonal and meridional wind components:

puv (u, v), but want marginal pdf of wind speed pw (w)
1. move from (u, v) to polar coordinates (w, ):
(u, v) = (w cos , w sin )
2.

w u u cos w sin

=

w v v sin w cos
df
dx
dx
px (x) dx = py (y) dy = py (y)

1
px (x) = exp Li xi
Z
EOF1
16oN
8oN
o
0
o
8 S
o
16 S
EOF2
o
150 E
180 W
150 W
0.2
0.4
120 W
0.6
90 W
16 N
8oN
o
0
o
8 S
16oS
0.8
0.4
Antisymmetric Component a(1)

=w

3. Integrate over to get marginal:

Z
puv (w cos , w sin ) d
pw (w) = w
16 N
o
8 N
o
0
o
8 S
o
16 S
150 E
180 W
0.2
120 W
90 W
0.2
0.4
Symmetric Component a(2)

o
150oE
0.2
180oW
0
150oW
0.2
0.4
120oW
0.6
90oW
16 N
o
8 N
o
0
o
8 S
o
16 S
0.8
150oE
0.15
180oW
0.1
Standard Deviation
150oE
0.2
180oW
0.4
150oW
0.6
150oW
0.05
120oW
0.05
90oW
0.1
Skewness
16 N
8oN
o
0
o
8 S
16oS
150 W
120oW
90oW
0.8
o
16 N
8oN
o
0
o
8 S
16oS
150oE
0.5
180oW
150oW
0
120oW
0.5
90oW
1
Case study: Tropical Pacific SST
Case Study: 20 hPa geopotential height
3
2
PC
1
0
1
2
3
3
PC1
References
Gardiner, C.W. Handbook of Stochastic Methods, Springer
Monahan, A.H. & A. Dai, 2004. The spatial and temporal structure
of ENSO nonlinearity. J. Clim, 17, 3026-3036.
Monahan, A.H., J.C. Fyfe, & L. Pandolfo, 2003. The vertical
structure of wintertime climate regimes of the Northern Hemisphere
extratropical atmosphere. J. Clim, 16, 2005-2021.
von Storch, H. & F.W. Zwiers, Statistical Analysis in Climate
Research, Cambridge University Press
Wilks, D.S. Statistical Methods in the Atmospheric Sciences,
Academic Press

Monahan 1

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Monahan 1

Transféré par

Droits d'auteur :

Formats disponibles

Introduction

An Introduction to Probability and

Climate is what you expect, weather is what you get.

School of Earth and Ocean Sciences, University of Victoria

Probability: Basic concepts

Probabilities are used to characterise processes with indeterminate

Our main focus will be case of basic outcome space of continuous

Any given random process X is associated with a set of possible

Define differential probability of x falling in small volume dx

dP (x0 ) = P (x0 x x0 + dx)

Can also define cumulative distribution function

(additivity of mutually exclusive events)

Joint and marginal pdfs

Probability of A and B together is joint probability

The conditional probability P (A|B) measures the probability of A

For x RN , px (x) = px (x1 , x2 , ..., xN ) is the joint pdf of multiple

P (A|B) is a measure of the dependence of A and B

For x, y independent, joint pdf factors as product of marginals:

Expectation and moments

From properties of conditional distributions

P (A, B) = P (A|B)P (B) = P (B|A)P (A)

Provides formal model of knowledge acquisition of system

The pdf px (x) defines a measure of relative probabilities, leading to

Expectation and moments

Moments of sea surface winds

Two common higher-order (reduced) moments:

measures asymmetry of pdf

measures flatness of pdf

Covariance and correlation

Covariance and correlation

These are measures of dependence between x and y

cov(x, y) = E{(x E{x})(y E{y})}

(a) indep, uncorr

Covariance matrix & EOFs

There are many families of pdfs characterised by one or more

Efficiently described in terms of eigenvectors ek known as empirical

For a Gaussian scalar x

EOFs orthogonal basis in RN which efficiently partitions variance

Central limit theorem and law of large numbers

One reason that Gaussian distributions are important is the central

Random vector X RN is multivariate Gaussian when pdf is

For a multivariate Gaussian

all marginals are Gaussian

Another useful asymptotic result is the law of large numbers: for xn

becomes Gaussian with finite variance as n

That is: as n becomes very large, fluctuations in sum vanish

Maximum entropy pdfs

Suppose y = f (x) where f is invertible

Variable y is just another way of talking about x; intuitively expect

P (y1 < y < y2 ) = P (x1 < x < x2 )

measures information content of a pdf

if y1 = f (x1 ) and y2 = f (x2 ) conservation of probability

where Li are Lagrange multipliers of constrained optimisation

Change of variables: example

pw (w, ) = wpuv (w cos , w sin )

Case study: Tropical Pacific SST

Know the joint pdf of zonal and meridional wind components:

px (x) dx = py (y) dy = py (y)

Antisymmetric Component a(1)

3. Integrate over to get marginal:

Symmetric Component a(2)

Case study: Tropical Pacific SST

Case Study: 20 hPa geopotential height