Random Processes

Random Process
Definition of Random Process
Consider we want to determine the probability of
temperature at 12 pm in certain city
The RV is X = temperature at 12 pm in city A
We have to record the data for many days to get p
X
(x)
However, the temperature is a function of time
The temperature varies between time to time and days to days
The RV = temperature in city A is function of time
An RV that is a function of time (or any other variable) is
called a random process or stochastic process
Definition of Random Process
The collection of all
possible waveform is known
as ensemble of the random
process x(t)
A waveform in the
collection is called a sample
function
Sample-function amplitudes
at some instant are the
values taken by the RV in
various trials
Definition of Random Process
Definition of Random Process
A discrete-time stochastic process is one when the index
set is a countable set
When the index set is continuous, it is called continuous-
time stochastic process
Example:
Definition of Random Process
The number of waveforms in an ensemble may be finite
or infinite
Example of infinite waveform: temperature
Example of finite waveform: output of binary signal
generator
Consider that we will examine the output over the period 0 to
2T
We will have only 2
2
= 4 waveforms
Definition of Random Process
Note that the randomness occurs to the selection of
waveforms
The waveforms (in the ensemble) themselves are
deterministic
Example: In the experiment of tossing a coin four times,
there are 16 possible outcomes. The randomness is which
of the 16 outcomes will occur in a given trial
Specification of A Random Process
How to specify random process:
Analytical
Experimental
Analytical: mathematical expression
Experimental:
We need to find some quantitatibe measure
Random process can be seen as collection of infinite,
independent number of RV
We need pdf
Autocorrelation
The autocorrelation function is defined as
( ) ( ) ( )
1 2 1 2 1 2
,
X
R t t x t x t x x = =
Example
A random process X(t) is defined by
where Y is a discrete random variable with P(Y = 0) =
and P(Y = /2) = . Find
X
(1) and R
XX
(0,1)
Solution:
X(1) is a random variables with values and
With probability each
( ) ( )
2cos 2 X t t Y t = +
( )
2cos 2t
( )
2cos 2 2 t t +
( )
1
1 2
2
P X ( = =

( )
1
1 0
2
P X ( = =

Example (Contd)
Thus,
The autocorrelation will be searched for t = 0 and t = 1
The values of P[X(0) = 0, X(1) = 0] =
P[X(0) = 2, X(1) = 2] =
Thus, the autocorrelation is
( ) ( ) { }
1 1
1 1 2 0 1
2 2
X
E X = = + =
( ) ( ) ( ) { }
( )( ) ( )( )
0,1 0 1
1 1
2 2 0 0 2
2 2
XX
R E X X =
= + =
Classification of Random Process
Stationary and Nonstationary random process
A random process whose statistical characteristics do not
change with time
Random process X(t) and X(t+) have the same statistical
properties
In other words, time translation of a sample function results in
another sample function of the random process having the
same probability
Example of stationary random process: noise
If the statistical characteristics depend on time, it is called
nonstationary random process
Example of nonstationary random process: temperature
Classification of Random Process
Wide-sense stationary process (WSS)
The mean and autocorrelation function do not vary with a
time shift
All stationary processes are WSS, but not vice versa
( ) { }
constant E X t K = =
( ) ( )
1 2 2 1
, ,
X X
R t t R t t t t = =
Classification of Random Process
Ergodic
Ensemble averages are equal to time averages of any sample
function
( ) ( )

x t x t =
( ) ( )
X X
R t t = 9
Classification of Random Process
Example
Show that , where and are
constants and is a random variable that is uniformly
distributed over (0, 2), is ergodic
Solution:
The ensemble average is
The time average is
( ) ( )
0
cos x t A t e u = + A
0
e
u
( ) ( ) ( )
2
0
0
1
cos 0
2
x x f d A t d
t
u
u u u e u u
t

( ( = = + =
} }
( ) ( )
0
0
0 0
1
cos 0
T
x t A t
T
e u = + =
}
Power Spectral Density of A Random
Process
PSD depicts the power spread in frequency domain
It is easy to get the frequency domain for deterministic
signal by using Fourier transform
How to compute the PSD for random signal?
Random signals are power signals
Several questions of determining the PSD for random
process arise:
We may not be able to describe the sample function
analytically
Every sample function may be different from another one
PSD is defined for stationary (or WSS)
Power Spectral Density of A Random
Process
We have to define the PSD of random process as the
ensemble average of the PSDs of all sample functions
Suppose that represents a sample function of a
random process . The truncated version of this sample
function is obtained by multiplying the signals with
rectangular function
( )
,
i
x t
( )
x t
( )
rect t T
( )
( )
, , 0.5
,
0,
i
T i
x t t T
x t
elsewhere

<
=

Power Spectral Density of A Random

Process
The Fourier transform is
The normalized energy in time interval (-T/2, T/2)
( ) ( )
( )
2
2
2
2
, ,
,
j ft
T i T i
T
j ft
T i
T
X f x t e dt
x t e dt
t
t

=
=
}
}
( ) ( )
2
2
T T T
E x t dt X f df


= =
} }
Power Spectral Density of A Random
Process
The mean normalized energy is obtained by taking the
ensemble average
The normalized average power is the energy expended
per unit time
( ) ( ) ( )
2
2
2 2
2
T
T
T T
T
E x t dt x t dt X f df


= = =
} } }
( ) ( )
( ) ( )
2
2 2
2
2
2
1 1
lim lim
1
lim
T
T
T T
T
T
T
P x t dt x t dt
T T
X f df x t
T

= =
(
= =
(

} }
}
Power Spectral Density of A Random
Process
The definition of PSD is
So that
( )
x
P S f df

=
}
( )
( )
2
lim
T
x
T
X f
S f
T

| |
|
=
|
|
\ .
Power Spectral Density of A Random
Process
Wiener-Khintchine Theorem
When is a wide-sense stationary process, the PSD can be obtained
from the Fourier transform of the autocorrelation function
( )
x t
( ) ( )
x x
R S f t
( ) ( ) ( )
2 j ft
x x x
S f F R f R e d
t
t t

( = =
}
( ) ( ) ( )
1 2 j ft
x x x
R F S f S f e df
t
t

( = =
}
Power Spectral Density of A Random
Process
Properties of PSD
PSD is always real
Let , we have
Therefore, the PSD is even function when is real
( ) ( ) ( )
x
R x t x t t t = +
( ) ( ) ( )
x
R x t x t t t =
t t o =
( ) ( ) ( ) ( )
x x
R x x R t o o t t = + =
( )
x t
Power Spectral Density of A Random
Process
Properties of PSD (Contd)
When is wide-sense stationary
( )
x t
( ) ( )
2
0
x x
S f df P x R

= = =
}
White Noise Processes
A random process is said to be a white noise process
if the PSD is constant over all frequencies
where is a positive constant
The autocorrelation function for the white noise process
is obtained by taking the inverse Fourier transform
( )
x t
( )
0
2
x
N
S f =
0
N
( ) ( )
0
2
x
N
R t o t =
Example
Determine the autocorrelation and the power of a low-
pass random process with white noise PSD
Solution:
From the figure we have
( )
2
X
N
S e =
( )
2 4
x
N
S f rect
B
e
t
| |
=
|
\ .
( ) ( )
sinc 2
x
R NB B t t t =
Example (Contd)
We calculate the power
We can also calculate the power by using integral
( )
2
0
x x
P x R NB = = =
( )
0
0
2
2
2
x x
B
P S df
N
df
NB
e

=
=
=
}
}
The Gaussian Random Process
A random process is said to be Gaussian if the
random variables have an N-
dimensional Gaussian PDF for any N and any
The N-dimensional Gaussian PDF can be written
compactly by using matrix notation
Let x be the column vector denoting the N random
variables
( )
x t
( ) ( ) ( )
1 1 2 2
, , ,
N N
x x t x x t x x t = = =
1 2
, , ,
N
t t t
( )
( )
( )
1 1
2 2
N N
x x t
x x t
x x t
(
(
(
(
(
(
= =
(
(
(
(
(


x

The Gaussian Random Process
The N-dimensional Gaussian PDF is
where the mean vector is
( )
( )
( ) ( ) ( )
1
1 2
2 1 2
1
2
T
N
f e
Det t

(

(

=
x m C x m
X
x
C
1
1
2
2
N
N
x
m
m
x
m
x
(
(
(
(
(
(
= = =
(
(
(
(
(


m x

The Gaussian Random Process

The covariance matrix C is defined by
where
For WSS random process
11 12 1
21 22 2
1 2
N
N
N N NN
c c c
c c c
c c c
(
(
(
=
(
(

C

( )( ) ( ) ( )
ij i i j j i i j j
c x m x m x t m x t m
(
( = =


( ) ( )
i i j j
m x t m x t m = = = =
The Gaussian Random Process
The elements of the covariance matrix is then
If happen to be uncorrelated for the
covariance matrix becomes
where
( )
2
ij x j i
c R t t m =
i
x
i j i j
x x x x =
i j =
2
2
2
0 0
0 0
0 0
o
o
o
(
(
(
=
(
(
(

C

( )
2 2 2 2
0
x
x m R m o = =
The Gaussian Random Process
Some properties of Gaussian processes:
depends only on C and on m, which is another way of
saying that the N-dimensional Gaussian PDF is completely
specified by the first- and second-order moments
Since the are jointly Gaussian, the are
individually Gaussian
When C is a diagonal matrix, the random variables are
uncorrelated
A linear transformation of a set of Gaussian random variables
produces another set of Gaussian random variables
A WSS Gaussian process is also strict-sense stationary
( )
X
f x
( )
i i
x x t ( =

( )
i i
x x t =
The Gaussian Random Process
Theorem:
If the input to a linear system is a Gaussian random
process, the system output is also a Gaussian process
Bandpass Random Process
Bandpass waveform can be represented by
or
and
where
( ) ( )
{ }
Re
c
j t
v t g t e
e
=
( ) ( ) ( )
cos sin
c c
v t x t t y t t e e =
( ) ( ) ( )
cos
c
v t R t t t e u ( = +

( ) ( )
( )
( ) ( )
j g t
g t g t e x t jy t
Z
= = +
Bandpass Random Process
The spectrum of bandpass waveform is
In communication systems, the random processes may be
random signals, noise, or signals corrupted by noise
If is Gaussian process, , , and are Gaussian
processes since they are linear functions of
But, and are not Gaussian because they are
nonlinear functions of
( ) ( ) ( )
*
1
2
c c
V f G f f G f f ( = +

( )
v t ( )
g t
( )
x t ( )
y t
( )
v t
( )
R t ( )
g t Z
( )
v t
Bandpass Random Process
Theorem:
If and are jointly WSS processes, the real bandpass
process will be WSS if
and only if
( )
x t ( )
y t
( ) ( )
{ }
( ) ( )
Re cos sin
c
j t
c c
v t g t e x t t y t t
e
e e = =
( ) ( )
( ) ( )
( ) ( )
0
xy yx
x y
x t y t
R R
R R
t t
t t
= =
=
=
Bandpass Random Process
Theorem:
If and are jointly WSS processes, the real bandpass
process
will be WSS when is an independent random variable
uniformly distributed over (0,2)
( )
x t ( )
y t
( ) ( )
( )
{ }
( ) ( ) ( ) ( )
Re cos sin
c c
j t
c c c c
v t g t e x t t y t t
e u
e u e u
+
= = + +
c
u