Kuliah 3b Siskom RandomProcesses PDF

RV PMF Cont. RV PDF Operations MRV Joint Dist.
Process Operations Types Quad filts Noise DC DC
COMS4100/7105:
Digital Communications
Lecture 3: Random Processes
Mandar Gujrathi
Bldg 78, Room 312
August 2, 2011
RV PMF Cont. RV PDF Operations MRV Joint Dist. Process Operations Types Quad filts Noise DC DC
Overview
• Random Variables
• Random Processes
• Noise
• Signal to Noise ratio
• Transmission and Filtering
• Digital Communications.
• Formatting analog information.

Random Variables
Discrete Random Variables
• Example: S= { Tail, Head}, Sx = {0, 1}
• A function that maps S into Sx is called as a Random Variable,
denoted by X (.)
• Sx is countable: Discrete Random Variable
• One to one mapping, Many to one mapping
• Sx is uncountable or infinite: Continuous Random Variable

One to one mapping
X (s2 )
s 2 Tail
x1  1
x2  0
s1 Head
S X  {1,0}
X (s1)
Sample Space S
• Example: S= { Tail, Head}, Sx = {1,0}

Probability of Random Variables
• The random variable X(.) maps on an event si from a sample space
S to xi in the sample space Sx
• In one to one mapping its basically the same event with different
names.
Hence, P[X (s) = xi ] = P[sj : X (sj ) = xi ] = P{si }
Probability of Random Variables
• For many to one mapping
P[X (s) = xi ] = P[sj : X (sj ) = xi ]

X
= P[sj ]
j:X (sj )=xi
fX [xi ] = P[X (s) = xi ]
• fX [xi ] is called the Probability Mass Function

Continuous Random Variable

• Example: Length of waiting times at an airport
• Sx is infinite and uncountable: Continuous R.V.
• Difficult to assign a specific probability to each value of X
• But we can calculate the probability of X lying in an interval.
exponential distribution
0.35
0.3
0.25
0.2
px(x)
0.15
0.1
0.05
0
0 a 5 10 15 20 25 30
b x
Z b
P[a ≤ X ≤ b] = fX (x)dx
a
Probability Density function (PDF)

• fX (x) is called as the PDF, fX [xi ] is called the PMF.
• Properties
• PDF/ PMF must be non negative
fX (x) ≥ 0
• PMF must sum to 1

M
X
fX [xi ] = 1
i=1
X∞
fX [xi ] = 1
i=1
• PDF must integrate to 1

Z ∞
fX (x)dx = 1
−∞
Expectation
• Expectation E [X ], Mean µ
X
E [X ] = xi fX [xi ]
i
Z ∞
= xfX (x)dx
−∞
Z ∞
= g (x)fX (x)dx
−∞
• Expectation is linear
E [a1 X1 + a2 X2 ] = a1 E [X1 ] + a2 E [X2 ]

nth moment
• Expectation E [X n ].
X
E [X ] = xin fX [xi ]
i
Z ∞
= x n fX (x)dx
−∞
• Variance, σ 2
Var (X ) = E [(X − µ)2 ]
σ 2 = E [(X − E [X ])2 ]
Z ∞
= (X − E [X ])2 fX (x)dx
−∞
• Variance is a non-linear operation

Two Random Variables
HH
1
TH
TT
1
HT x
S XY
S
Concept of Multiple Random Variable
• Outcome of 2 coin tosses = {HT , TH, TT , HH}
• To map these events we require 2 random variables.
• But
 we are
mapping
 from the same sample space
X (si ) x
S  =  i
Y (si ) yi
• Two random variables defined on the same sample space are Jointly
distributed.       
1 0 0 1

• SX ,Y =   ,   ,   ,  
0 1 0 1
• These are discrete events, so joint PMF
Joint PMF
• For a single RV, PMF: fX [xi ] = P[X (s) = xi ]
• For a multiple RV, Joint PMF:
fX ,Y [xi , yj ] = P[X (s) = xi , Y (s) = yj ],i = 1, . . . , Nx
j = 1, . . . , Ny
• Properties of Joint PMF
0 ≤ fX ,Y [xi , yj ] ≤ 1
Ny
Nx X
X
fX ,Y [xi , yj ] = 1
i=1 j=1
Joint distributions
r =1
Sample Space S
Joint distributions
r =1 (r , θ)
Sample Space S New Sample Space Sr ,θ
Sr ,θ = {(r , θ) : 0 ≤ r ≤ 1, 0 ≤ θ ≤ 2π}
Random Processes
Random Signals and Process
• Random Variables
X (s2 )
s 2 Tail
x1  1
x2  0
s1 Head
S X  {1,0}
X (s1)
Sample Space S
Random process example
• Mapping the outcomes from the original experimental sample space.
S = {(H, H, T , . . . ), (H, T , H, . . . ), (T , T , H, . . . ), . . . }
SX = {(1, 1, 0, . . . ), (1, 0, 1, . . . ), (0, 0, 1, . . . ), . . . }
= {x1 , x2 , x3 , . . . }
• with each outcome of the random process denoted as
x1 = (x[0], x[1], . . . )
• The random process is a mapping from S which is a set of infinite
and sequential experimental outcomes to SX which is an infinte

sequence of realisations.
Random process
1.5
X[n,s1] = x1[n]
x1[n] 1
0.5
0
0 2 4 6 8 10 12 14 16 18 20
n
1.5
X[n,s2] = x2[n]
1
x2[n]
0.5
0
0 2 4 6 8 10 12 14 16 18 20
n
1.5
X[n,s3] = x3[n]
1
x3[n]
0.5
0
0 2 4 6 8 10 12 14 16 18 20
n
Random process
• A sample space or ensemble composed of functions of time is called
Random or stochastic process

• Suppose s1 , s2 , . . . , sn form the sample points of a sample space S
X (t, s), − T ≤ t ≤ T
xj (t) = X (t, sj )
x1 ( t )
x1 ( t k )
s1
s2 x 2 (t ) x 2 (t k )
s3
Sample Space x3 (t )
Random process
• Is an ensemble composed of functions of time.
• For a single sample point s1 we get a single function of time x1 (t).
• If time is held constant t1 we get a random variable.
• For a given random process, the mean value of X (t) at arbitrary
time t is defined as
X (t) = E [X (t)]
• E [X (t)] is the ensemble average obtained by averaging over all the
sample functions with t held constant.

Ensemble and Time Averages
• The ensemble average at a given time t is defined as

Z ∞
E [X (t)] = xfX (t) (x)dx
−∞
• In practice, we often have access to a single realisation of stochastic
process
• In this case it is possible to define the time average of a signal

Z T
1 2
hx(t)i = lim x(t)dt.
T →∞ T − T2
Ergodic Process
• A process is said to be ergodic if all time averages are equal to the
corresponding ensemble averages.
• As per the definition,
E [X (t)] = hx(t)i = µx
2
= (x(t) − µx )2 = σx2

E X (t) − µx Variance
Random Process: Operations
• Autocorrelation
Rx (t1 , t2 ) = E [X (t1 )X (t2 )]

Z ∞Z ∞
= x1 x2 fX (t1 )X (t2 ) (x1 x2 )dx1 dx2
−∞ −∞
• Autocovariance
Cx (t1 , t2 ) = E [{X (t1 ) − µx (t1 )}{X (t2 ) − µx (t2 )}]
= RX (t1 , t2 ) − µx (t1 )µx (t2 )

Operations: Random Process
• Crosscorrelation
RXY (t1 , t2 ) = E [X (t1 )Y (t2 )]

Z ∞Z ∞
= xyfX (t1 )Y (t2 ) (xy )dxdy
−∞ −∞
• Cross covariance
CXY (t1 , t2 ) = E [{X (t1 ) − µx (t1 )}{Y (t2 ) − µy (t2 )}]
= RXY (t1 , t2 ) − µx (t1 )µy (t2 )

Operations
• The two random processes are said to be uncorrelated if
CXY (t1 , t2 ) = RXY (t1 , t2 ) − µx (t1 )µy (t2 ) = 0
RXY (t1 , t2 ) = µx (t1 )µy (t2 )
RXY (t1 , t2 ) = E [X (t1 )]E [Y (t2 )].
• For a complex Stochastic Process, autocorrelation is
Z (t) = X (t) + jY (t)
RZ (t1 , t2 ) = E [Z (t1 )Z ∗ (t2 )]

Types of Random Processes

Strict Sense Stationary Process
• A random process X (t). At times t1 , t2 , . . . , tn , we observe random
variables X (t1 ), X (t2 ), . . . , X (tn ).
• The joint p.d.f is fX (t1 ),X (t2 ),...,X (tn ) x1 x2 . . . xn .
• A random process is strict sense stationary if
fX (t1 +τ ),X (t2 +τ ),...,X (tn +τ ) (x1 x2 . . . xn ) =
fX (t1 ),X (t2 ),...,X (tn ) (x1 x2 . . . xn )
• The joint p.d.f of random variables obtained by observing the
random process is invariant to time shift.

IID random process
• So is IID random process stationary?
• Yes, Marginal PDF is the same for each RV. Therefore
fX (t1 +τ ),X (t2 +τ ),...,X (tn +τ ) (x1 x2 . . . xn ) =
fX (t1 ),X (t2 ),...,X (tn ) (x1 x2 . . . xn )
• Any process whose mean or variances change with time is NOT
stationary.
Wide sense Stationary Process
• Using joint p.d.f, first order distribution function, i.e. for n = 1 will
be
fX (t1 +τ ) (x) = fX (t1 ) (x)
• If t1 = 0, then
fX (τ ) = fX (0) (x) ∀τ
• Since the PDF does not depend on a particular time, so should not
mean/expectation. Therefore
Z ∞
E [X (t)] = xfX (x)dx = µx = constant
−∞
• Mean of a wide-sense stationary process is constant

Wide sense stationary process
• Now, for n = 2 we have:
fX (t1 +τ )X (t2 +τ ) (x1 x2 ) = fX (t2 )X (t1 ) (x1 x2 ) ∀t1 , t2
• If τ = −t1 we have,
fX (t1 −t1 )X (t2 −t1 ) (x1 x2 ) = fX (t2 )X (t1 ) (x1 x2 ) ∀t1 , t2
• This leads to
E [X (t1 )X (t2 )] = E [X (0)X (t2 − t1 )]
RX (t1 , t2 ) = RX (t2 − t1 )
• Such a process is also called weakly stationary.

Covariance
• Recall that autocovariance of a random process is
Cx (t1 , t2 ) = E [{X (t1 ) − µx (t1 )}{X (t2 ) − µx (t2 )}]
= RX (t1 , t2 ) − µx (t1 )µx (t2 )
• If it is WSS,
CX (t1 , t2 ) = RX (t2 − t1 ) − µ2x

To summarise
• A second order stationary process can be wide-sense stationary but
the converse is not true.
• A random process is ergodic if all time averages of sample functions
equal to corresponding ensemble averages.
• A random process is wide-sense stationary when the mean is
independent of time and the autocorrelation function depends on

the time difference.
• A random process is strictly stationary when the statistics do not
change regardless of any shift in time.
• A strict sense stationary process can be wide sense stationary but
the converse is not true.

Properties of correlation for WSS process
• An Autocorrelation function is defined as
RX (τ ) = E [X (t + τ )X (t)] ∀t
• Mean square value of the process can be obtained by τ = 0.
RX (0) = E [X 2 (t)] this is second moment

Properties
• Autocorrelation is an even function
RX (τ ) = RX∗ (−τ )
100
80
60
autocorrelation
40
20
−20
−40
−100 −80 −60 −40 −20 0 20 40 60 80 100
shift
>> x = randn(1,101);
>> [a, shift] = xcorr(x);
>> plot(shifts,a);
Properties of correlation
• Autocorrelation function RX (τ ) has the maximum magnitude at
τ =0
E [(X (t + τ ) + X (t))2 ] ≥ 0
E [X 2 (t + τ )] + E [X 2 (t)] + 2E [X (t + τ )X (t)] ≥ 0
2RX (0) + 2RX (τ ) ≥ 0
RX (0) ≥ |RX (τ )|
Joint stationary
• For jointly WSS processes,
∗
RXY (τ ) = RYX (−τ )
• If for all τ , RXY (τ ) = 0 we say X(t) and Y(t) are orthogonal
• If for all τ , CXY (τ ) = 0 we say X(t) and Y(t) are uncorrelated.

Example
• Consider a sinusoidal signal with random phase defined as
X (t) = X (t) cos(2πfc t + θ)
where A and fc are constants and θ is a random variable uniformly

distributed over the interval [−π, π] and independent of X (t).
Obtain the Rx (τ ) and PSD.
Homework
• Consider a pair of quadrature modulated process that are related to
stationary process as
X1 (t) = X (t) cos(2πfc t + θ)
X2 (t) = X (t) sin(2πfc t + θ)
where fc is a carrier frequency and θ is a random variable uniformly

distributed over the interval [0, 2π] and independent of X (t). Obtain
the cross correlation.
Filtering Stochastic Processes

Filtering Stochastic Processes
• Consider a pair of LTI filters and process as shown.
X (t) V (t) Y (t) Z (t)

h1 (t) h2 (t)
• The cross correlation and cross spectral densities can be written as
Rvz (τ ) = v (τ ) ∗ z(−τ )
= h1 (τ ) ∗ x(τ ) ∗ h2∗ (−τ ) ∗ y (−τ )
Rvz (τ ) = h1 (τ ) ∗ h2∗ (−τ ) ∗ Rxy (τ )
Gvz (f ) = H1 (f )H2∗ (f )Gxy (f )

Quadrature Filters
• A common tool in communications is the phase shifter, a device that
shifts the phase of its input by some number of degrees,

A cos(2πf0 t) A cos(2πf0 t − 90◦ )
−90◦ phase shifter
• Considered in terms of its constituent complex exponentials, this

phase shifter is shifting
• the phases of positive frequencies by −π/2 and
• the phases of negative frequencies by +π/2.

Quadrature Filters
• The transfer function of a Quadrature filter can be written as
HQ (f ) = −jsgn(f ) = −j f >0
= +j f <0
• Hence,
HQ ( f )
j
Quadrature Filters
• If we want to write the impulse response,
1
F[sgn(t)] =
jπf
1
F[] = sgn(−f ) = −sgn(f )
jπt
j 1
F −1 [−jsgn(f )] = =
jπt πt
If F(x(t)) = X (f ) then F(X (t)) = x(−f ) Duality
• Hence the impulse response is
1
hQ (t) = .
πt
Quadrature Filters and Hilbert Transforms
• Hence, given an input g (t), to produce a −90◦ -phase shifted output
ĝ (t), we perform the convolution

Z ∞
1 1 g (τ )
ĝ (t) = ∗ g (t) = dτ.
πt π −∞ t −τ
• The integral formula is known as the Hilbert transform and g (t) and
ĝ (t) as a Hilbert transform pair.
• The filter is known as a Hilbert transformer or a quadrature filter.

Quadrature Filters and Hilbert Transforms
• If ĝ (t) is the Hilbert transform of g (t) then −g (t) is the Hilbert
transform of ĝ (t).
• The inverse Hilbert transform is given by

Z ∞
1 ĝ (τ )
g (t) = − dτ.
π −∞ t −τ
• Observe that, if g (t) is real, so is ĝ (t). In this case, pairs are
orthogonal: Z ∞
g (t)ĝ (t)dt = 0.
−∞
Noise
• A communications signal at the receiver is usually modelled as a
stochastic process. We usually identify two components in the

process:
• the information-bearing component — the signal — and
• the component that bears no information — the noise.
• We sometimes identify a third component: a component that bears
unwanted information, that of another user — interference.

Thermal Noise
• Produced by random motion of charged particles in conducting
media.
• In 1928, Johnson & Nyquist studied noise observed in resistors
(Johnsons noise/ resistance noise).

• They found that the noise voltage can be modelled as Gaussian.
• Furthermore, it has a spectral density

2Rh|f |
GV (f ) = V 2 /Hz
e h|f |/kT − 1
• R is the resistance (Ω), T is the temperature (K),
• k = 1.38 × 10−23 J/K is Boltzmann’s constant,
• h = 6.62 × 10−34 J · s is Planck’s constant.
• It turns out that this density is almost flat up to infra-red f’s.

Thermal Noise
• This could be written as

h|f | kT
Gv (f ) = 2RkT 1 − |f | <<
2kT h
• Hence, we tend to use the (very accurate) approximation
GV (f ) = 2RkT V 2 /Hz.
White & Coloured Noise

• Thermal noise has (for our purposes) a flat spectral density.
• By analogy to visible light, we call such noise white noise.
• Its autocorrelation and PSD are
N0
RW (τ ) = δ(τ )
2
N0
GW (f ) =
2
RW (τ ) = 0 t 6= 0
• Hence, any two samples separated in time are uncorrelated.
• However, a theoretical problem is that white noise has infinite
average power.
• Filtered white noise is called coloured noise.
Coloured Noise
N0
• Suppose we pass a Gaussian white noise with SD 2 through an LTI
filter with transfer function H(f ).
• Resulting output is
N0
Gy (f ) = |H(f )|2
2
N0 −1
Ry (τ ) = F [|H(f )|2 ]
2
• SD of filtered noise takes the shape of |H(f )|2
• Hence, filtered white noise is called coloured noise.

Signal-to-Noise Ratio
• Typically, the signal and noise components are added together in a

received communication signal.
• In additive noise our received process Y (t) is
Y (t) = S(t) + N(t)
S(t) is the noise-free signal (or process) and N(t) is the noise.
• We usually assume that:
• the noise is ergodic and zero-mean,

• the noise is independent of the signal.
Signal-to-Noise Ratio
• With E [|S(t)|2 ] = PS and E [|N(t)|]2 = PN we find that
E [|Y (t)|2 ] = PS + PN
• so the signal and noise power add also in the received signal.
• An important quantity is the signal-to-noise ratio (SNR):
SNR = Ps /PN , often quoted in dB.
• A further typical assumption is that the noise is white and Gaussian,
hence Additive White Gaussian Noise (AWGN).

Kuliah 3b Siskom RandomProcesses PDF

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Kuliah 3b Siskom RandomProcesses PDF

Transféré par

Droits d'auteur :

Formats disponibles

RV PMF Cont. RV PDF Operations MRV Joint Dist.

Process Operations Types Quad filts Noise DC DC

Lecture 3: Random Processes

Bldg 78, Room 312

• Signal to Noise ratio

• Transmission and Filtering

• Formatting analog information.

Discrete Random Variables

• Example: S= { Tail, Head}, Sx = {0, 1}

• A function that maps S into Sx is called as a Random Variable,

• Sx is countable: Discrete Random Variable

• One to one mapping, Many to one mapping

• Sx is uncountable or infinite: Continuous Random Variable

One to one mapping

• Example: S= { Tail, Head}, Sx = {1,0}

Probability of Random Variables

• The random variable X(.) maps on an event si from a sample space

S to xi in the sample space Sx

Probability of Random Variables

• For many to one mapping

P[X (s) = xi ] = P[sj : X (sj ) = xi ]

fX [xi ] = P[X (s) = xi ]

• fX [xi ] is called the Probability Mass Function

Continuous Random Variable

• Sx is infinite and uncountable: Continuous R.V.

• Difficult to assign a specific probability to each value of X

• But we can calculate the probability of X lying in an interval.

Probability Density function (PDF)

• PMF must sum to 1

• PDF must integrate to 1

E [a1 X1 + a2 X2 ] = a1 E [X1 ] + a2 E [X2 ]

Var (X ) = E [(X − µ)2 ]

• Variance is a non-linear operation

Two Random Variables

Concept of Multiple Random Variable

• Outcome of 2 coin tosses = {HT , TH, TT , HH}

• To map these events we require 2 random variables.

• For a single RV, PMF: fX [xi ] = P[X (s) = xi ]

• For a multiple RV, Joint PMF:

fX ,Y [xi , yj ] = P[X (s) = xi , Y (s) = yj ],i = 1, . . . , Nx

• Properties of Joint PMF

Sample Space S New Sample Space Sr ,θ

Random Signals and Process

Random process example

• Mapping the outcomes from the original experimental sample space.

SX = {(1, 1, 0, . . . ), (1, 0, 1, . . . ), (0, 0, 1, . . . ), . . . }

• with each outcome of the random process denoted as

• The random process is a mapping from S which is a set of infinite

and sequential experimental outcomes to SX which is an infinte

Random or stochastic process

• Is an ensemble composed of functions of time.

• For a single sample point s1 we get a single function of time x1 (t).

• If time is held constant t1 we get a random variable.

• For a given random process, the mean value of X (t) at arbitrary

• E [X (t)] is the ensemble average obtained by averaging over all the

sample functions with t held constant.

Ensemble and Time Averages

• The ensemble average at a given time t is defined as

• In practice, we often have access to a single realisation of stochastic

• In this case it is possible to define the time average of a signal

• A process is said to be ergodic if all time averages are equal to the

corresponding ensemble averages.

• As per the definition,