INHA UNIVERSITY

Probability
and Random Variable
(Chapter 5)
Il-Suek Koh
ikoh@inha.ac.kr
Pairs of Random Variables
Marginal/Joint PMF/CDF/PDF
Independence of Two R.V.s
Joint Moments & Expected Value of Function of Two R.V.s
Conditional Probability & Conditional Expectation
Functions of Two Random Variables
Pairs of Jointly Gaussian Random Variables
Two Random Variables
Introduction
Techniques for calculating the probability of events involving a single R.V. in
Chapter 4.
Extend to two random variables!!
Joint pmf, cdf, and pdf
Expected value to define joint moments
Independence, correlation
Conditional probability involving a pair of R.V.s
Two R.V.s
Function that assigns a pair of real numbers X() = (X(),Y())
Vector function that maps S into R
2
, the real plane
S

x
y
X()
Two Random Variables
Consider following events
A = {X + Y 10}
B = {min(X,Y) 5}
C = {X
2
+ Y
2
100}
Those events are represented as follows:
B: {x y & y 5} {x < y & x 5}
y
x
(0,10)
(10,0)
A
x
y
(0,10)
(10,0)
C
x
B
y
(5,5)
Pairs of Discrete Random Variables
Let vector R.V. X = (X,Y)
Sample space S
X,Y
= {(x
j
,y
k
), j=1,2,, k=1,2,}
Joint pmf of X
Probability of any event B
If B = S
X,Y
, then
( ) { } { }
( )
,
,
,
, ,
X Y j k j k
j k j k X Y
p x y P X x Y y
P X x Y y for x y S
(
= = =

( = = e

| | ( )
( )
,
,
,
j k
X Y j k
in B x y
P in B p x y =

X
( )
,
1 1
, 1
X Y j k
j k
p x y

= =
=

Pairs of Discrete Random Variables

Marginal pmf
Joint pmf: provide information about joint behavior of X & Y
Extract probability of event involving each of R.V. from joint pmf
( )
( )
1 2
,
1
,
, ,
,
X j j
j
j j
X Y j k
k
p x P X x
P X x Y anything
P X x Y y P X x Y y
p x y

=
( = =

( = = =

( ( = = = = =

=

( ) ( )
,
1
,
Y k X Y j k
j
p y p x y

=
=

Joint CDF
Joint CDF
Probability of event {X x
1
}{Y y
1
}
Amount of mass contained in the rectangular region
Probability of semi-infinite rectangle defined by (x
1
,y
1
)
( ) | |
, 1 1 1 1
, ,
X Y
F x y P X x Y y = s s
x
y
(x
1
,y
1
)
F
X,Y
(x
1
,y
1
) = P[X x
1
,Y y
1
]
Joint CDF
Properties of Joint CDF
Nondecreasing function of x and y
F
X,Y
(x
1
,y
1
) F
X,Y
(x
2
,y
2
) if x
1
x
2
& y
1
y
2
F
X,Y
(x
1
,) = 0, F
X,Y
(,y
1
) = 0, F
X,Y
(,) = 1
Marginal cdf
F
X
(x
1
) = F
X,Y
(x
1
,), F
Y
(y
1
) = F
X,Y
(,y
1
)
Continuous from north and east
and
Probability of rectangular {x
1
< x x
2
, y
1
< y y
2
} is given by
P[x
1
< x x
2
, y
1
< y y
2
] = F
X,Y
(x
2
,y
2
)F
X,Y
(x
2
,y
1
)F
X,Y
(x
1
,y
2
)+F
X,Y
(x
1
,y
1
)
( ) ( )
, ,
lim , ,
X Y X Y
x a
F x y F a y
+

= ( ) ( )
, ,
lim , ,
X Y X Y
y b
F x y F x b
+

=
Joint PDF
Joint cdf
Probability of events corresponding to rectangular shapes
Reasonable shape case??
Disk, polygon, half-plane
Approximate to union of disjoint infinitesimal rectangles
x
y
B
j,k
x
y
B
j,k
| | ( )
( )
, ,
,
,
j k
j k X Y j k
j k B x y
P B P B f x y x y
e
( ~ = A A


density function
Joint PDF
Joint pdf
When x, y 0,
where f
X,Y
(x,y) called joint probability density function
Note
Relationship between joint cdf and joint pdf (continuous case)
| | ( )
,
', ' ' '
X Y
B
P inB f x y dx dy =
}}
X
( )
,
', ' ' ' 1
X Y
f x y dx dy


=
} }
( ) ( )
, ,
, ', ' ' '
y x
X Y X Y
F x y f x y dx dy

=
} }
( )
( )
2
,
,
,
,
X Y
X Y
F x y
f x y
x y
c
=
c c
Joint PDF
Probability of Rectangle Region
B = {(x,y): a
1
< x b
1
and a
2
< y b
2
}
Probability of Infinitesimal Rectangle
Marginal PDF
| | ( )
2 1
2 1
1 1 2 2 ,
, ,
b b
X Y
a a
P a X b a Y b f x y dxdy < s < s =
} }
| | ( )
( )
,
,
, ', ' ' '
,
y dy x dx
X Y
y x
X Y
P x X x dx y Y y dy f x y dx dy
f x y dxdy
+ +
< s + < s + =
~
} }
( )
( ) ( )
( )
{ }
( )
,
,
,
,
', ' ' '
, ' '
x
X Y X
X X Y
X Y
dF x dF x
d
f x f x y dy dx
dx dx dx
f x y dy

= = =
=
} }
}
( ) ( )
,
', '
Y X Y
f y f x y dx

=
}
Joint PDF
Example 5.15 Jointly Uniform R.V.s
1. x < 0 & y < 0
2. 0 x < 1 & 0 y < 1
3. 0 x < 1 & y > 1
4. x > 1 & 0 y < 1
5. x > 1 & y > 1
,
1 0 1 0 1
0
X Y
x and y
f
elsewhere
s s s s

x
y
1 2
3
4
5
(1,1)
( )
,
, 0
X Y
F x y =
( )
,
0 0
, 1 ' '
y x
X Y
F x y dx dy xy = =
} }
( )
1
,
0 0
, 1 ' '
x
X Y
F x y dx dy x = =
} }
( )
1
,
0 0
, 1 ' '
y
X Y
F x y dx dy y = =
} }
( )
1 1
,
0 0
, 1 ' ' 1
X Y
F x y dx dy = =
} }
Independence of Two Random Variables
Definition
If any event A
1
defined in terms of X is independent of any event A
2
defined in terms of Y, then X and Y are independent random variables.
Determining whether X and Y are independent.
Using PMF
Using CDF
Using PDF
| | | | | |
1 2 1 2
, P X in A Y in A P X in A P Y in A =
( ) ( ) ( )
,
,
X Y j k X j Y k
p x y p x p y =
( ) ( ) ( )
,
,
X Y X Y
F x y F x F y =
( ) ( ) ( )
,
,
X Y X Y
f x y f x f y =
Independence of Two Random Variables
Example 5.18 and 5.22
Joint Gaussian Random Variable
Ex) m
X
= 1, m
Y
= 3, VAR[X] = 1, VAR[Y] = 3, = 0
-3 -2 -1 0 1 2 3 4 5
-1
0
1
2
3
4
5
6
7
-4
-2
0
2
4
6
-5
0
5
10
0
0.02
0.04
0.06
0.08
0.1
( )
( ) ( )
2 2 2
2 2 1
,
2
1
,
2 1
x xy y
X Y
f x y e

t
+
=

Independence of Two Random Variables

Example 5.18 and 5.22
Find marginal pdf of X and Y
Are X and Y independent?
X and Y are independent iff = 0
: correlation coefficient
( )
( )
( )
( )
2
2
2 2
2 1
2 2
2
2 2
2 1
y x
x x
X
e e e
f x dy

t t
t

= =

}
( )
( )
( )
( )
2
2
2 2
2 1
2 2
2
2 2
2 1
x y
y y
Y
e e e
f x dx

t t
t

= =

}
( )
( )
( ) ( )
2 2
2
,
1
,
2
x y
X Y X Y
f x y e f x f y
t
+
= = iff = 0
Independence of Two Random Variables
Consider any functions of R.V.s
Let g(X) and h(Y)
If X & Y are independent g(X) & h(Y) are also independent
Proof)
A: event involving g(X)
A': equivalent event involving X
B: event involving h(Y)
B': equivalent event involving Y
( ) ( ) | |
| | | |
( ) ( )
, ', '
' '
P g X in A h Y in B P X in A Y in B
P X in A P Y in B
P g X in A P h Y in B
( =

=
( ( =

Joint Moments and Expected Value
Expected Value of a Function of Two R.V.s
Let Z = g(X,Y)
Example 5.24
Let Z = X + Y
| |
( ) ( )
( ) ( )
,
,
, ,
,
, , ,
X Y
i n X Y i n
i n
g x y f x y dxdy
X Y jointly contunuous
E Z
X Y discrete g x y p x y

} }

| | ( ) ( )
( ) ( )
( ) ( ) | | | |
,
, ,
,
, ,
X Y
X Y X Y
X Y
E Z x y f x y dxdy
xf x y dydx yf x y dxdy
xf x dx yf y dy E X E Y






= +
= +
= + = +
} }
} } } }
} }
Joint Moments and Expected Value
Example 5.25
Let g(X,Y) = g
1
(X)g
2
(Y), X and Y are independent.
Expected Value of Sum of n R.V.s
R.V.s do not have to be independent!
( ) ( ) ( )
( ) ( ) ( )
( ) ( )
{ }
( ) ( )
{ }
( ) ( )
1 2
1 2 ,
1 2
1 2
,
,
X Y
X Y
E g X Y E g X g Y
g x g y f x y dxdy
g x f x dx g y f y dy
E g x E g Y




( ( =

=
=
( ( =

} }
} }
| | | | | | | |
1 2 1 2 n n
E X X X E X E X E X + + + = + + +
Joint Moments and Expected Value
Joint Moments, Correlation, Covariance
jkth joint moment of X and Y
Correlation (j = k = 1 moment)
Orthogonal: E[XY] = 0
Central moment
Covariance (j = k = 1 central moment)
( )
( )
,
,
,
,
, ,
j k
X Y
j k
j k
i n X Y i n
i n
x y f x y dxdy
X Y jointly continuous
E X Y
X Y discrete x y p x y

( =

} }

| |
E XY
| | ( ) | | ( )
j k
E X E X Y E Y
(


( ) | | ( ) | | ( )
| | | | | |
, COV X Y E X E X Y E Y
E XY E X E Y
(
=

=
Joint Moments and Expected Value
Example 5.26
Covariance of independent R.V.s
Correlation Coefficient
1 1
Uncorrelated
X,Y
= 0
Two R.V.s are independent they are uncorrelated
Two R.V.s are uncorrelated Are they independent???
( ) | | | | | |
| | | | | | | |
,
0
COV X Y E XY E X E Y
E X E Y E X E Y
=
= =
( )
( ) ( )
| | | | | |
,
,
X Y
X Y
COV X Y E XY E X E Y
STD X STD Y

o o

= =
Conditional Probability
Recall Conditional Probability
Case 1: X is Discrete R.V.
1) Y is discrete R.V.
Conditional pmf of Y given X = x
Independent X and Y:
Total Probability
| |
| |
| |
,
0
P Y in A X x
P Y in A X x for P X x
P X x
=
( = = = >

=
( )
| |
| |
( )
( )
,
, ,
X Y
Y
X
p x y P X x Y y
p y x P Y y X x
P X x p x
= =
( = = = = =

=
( ) ( )
Y j k Y j
p y x p y =
| | ( )
k
k X k
x
P Y in A P Y in A X x p x

( = =

Conditional Probability
Case 1: X is Discrete R.V.
2) Y is continuous R.V.
Conditional cdf of Y given X = x
k
Conditional pdf of Y given X = x
k
Independence
( )
| |
| |
,
k
Y k
k
P Y y X x
F y x
P X x
s =
=
=
( ) ( )
Y k Y k
d
f y x F y x
dy
=
( ) ( )
Y Y
F y x F y =
( ) ( )
Y Y
f y x f y =
Conditional Probability
Example 5.31 Binary Communication System
Y = X + N, N: zero-mean, unit variance Gaussian R.V.
Find f
Y
(y | x = +1), f
Y
(y | x = 1), P(X = +1 | Y > 0)
If X = +1, Y = N + 1
If X = 1, Y = N 1
TX RX
X Y
N
| |
1 1/ 3 P X = + =
| |
1 2 / 3 P X = =
( )
( )
2
1 2
1
2
x
Y
e
f y X
t

= + =
( )
( )
2
1 2
1
2
x
Y
e
f y X
t
+
= =
( )
2
2
2
n
N
e
f n
t

=
P(X = +1 | Y > 0)
( )
( )
( )
0 1 / 3
1 0
0
P Y X
P X Y
P Y
> = +
= + > =
>
( )
( ) ( )
0
0 1 2 0 1
3 3
P Y
P Y X P Y X
>
> = + > =
= +
( )
( )
1 1
1 0 0.726
1 1
Q
P X Y
Q

( = + > = =

+
Conditional Probability
Case 2: X is Continuous R.V.
P[X = x] = 0 for continuous X
Define: Conditional cdf of Y given X = x
Conditional pdf of Y given X = x
( ) ( )
| |
| |
( )
( )
( )
( )
0
0
, ,
0 0
lim
,
lim
', ' ' ' , ' '
lim lim
' '
Y Y
h
h
y x h y
X Y X Y
x
x h
h h
X
X
x
F y x F y x X x h
P Y y x X x h
P x X x h
f x y dx dy f x y dy h
f x h
f x dx

+

+

= < s +
s < s +
=
< s +
= =
} } }
}
( )
( )
( )
,
, ' '
y
X Y
Y
X
f x y dy
F y x
f x

=
}
( )
( )
( )
,
,
X Y
Y
X
f x y
f y x
f x
=
Conditional Probability
Case 2: X is Continuous R.V.
Independence between X and Y
Total Probability Theorem
Example 5.35
( ) ( ) ( )
,
,
X Y X Y
f x y f x f y =
( ) ( )
X X
f x y f x =
( ) ( )
Y Y
f y x f y =
( ) ( ) ( )
,
,
X Y X Y
F x y F x F y =
( ) ( )
X X
F x y F x =
( ) ( )
Y Y
F y x F y =
| | ( )
X
P Y in A P Y in A X x f x dx

( = =
}
| |
| |
( ) ( )
( ) ( ) ( ) ( )
2
1 1
1 1
1 1 3
1
1 2 1 1 3 1 2 3
Y
y
Y Y
P X Y y P X y Y y h
P y Y y h X P X
P y Y y h
f y h
e f y h f y h

( ( = + = = = + < s +

( < s + = + = +

=
< s +
+
= =
+ + +
Conditional Expectation
Conditional Expectation of Y given X
( )
Y
E Y X yf y x dy

( =
}
( )
j
k j Y j k
y
E Y x y p y x ( =


Function of x
( )
( ) ( )
( )
( ) | |
,
,
X
Y X
X Y
Y
E E Y X E Y X f x dx
yf y x dy f x dx
y f x y dxdy
yf y dy E Y

(
( ( =


=
=
= =
}
} }
} }
}
Expectation w.r.t. y
? E E Y X
(
( =


Expectation w.r.t. x
Conditional Expectation
Example 5.37
E[Y | X = +1] = 1 and E[Y | X = 1] = 1
E[Y ] = ?
| |
| |
| | | |
( )
1 1 1 1
1(1 3) 1 2 3 1 3
k
E Y E E Y X
E Y X P X k
E Y X P X E Y X P X
(
( =


( = =

( ( = = + = + + = =

= + =

Functions of Two Random Variables

One Function of Two R.V.s
Given two R.V.s X and Y and a function g(X,Y)
Define a new R.V. Z Z = g(X,Y)
Given f
X,Y
(x,y) find f
Z
(z)?
Firstly, find cdf of Z
Then, pdf of Z is found by taking derivative of F
Z
(z)
( ) | | ( )
( )
( )
,
,
,
', ' ' '
z
Z
X Y
x y R
F z P Z z P g X Y z
f x y dx dy
e
( = s = s

=
}}
( ) ( ) { }
, :
z
R x y g z = = s x x
( ) ( )
Z Z
d
f z F z
dz
=
Functions of Two Random Variables
One Function of Two R.V.s
Z = X + Y
Note
x
y
( ) | |
( )
( )
,
,
,
,
Z
z y
X Y
z x
X Y
F z P X Y z
f x y dxdy
f x y dydx




= + s
=
=
} }
} }
z-y
x
y
z-x
( ) ( )
( )
( )
,
b z
Z
a z
F z f x z dx =
}
( )
( ) ( )
( ) ( )
( )
( ) ( )
( )
( )
( )
,
,
,
Z
Z
b z
a z
dF z db z
f z f b z z
dz dz
da z f x z
f a z z dx
dz z
= =
c
+
c
}
Functions of Two Random Variables
One Function of Two R.V.s
Z = X + Y
Integration along x axis
Integration along y axis
( ) ( )
( )
( ) ( )
( )
,
, ,
,
,
, 0 ,
,
z y
Z X Y
z y
X Y X Y
X Y
f z f x y dx dy
z
f z y y f x y dx dy
z
f z y y dy

c
=
c
c
| |
= +
|
c
\ .
=
} }
} }
}
( ) ( )
( )
( ) ( )
( )
,
, ,
,
,
, 0 ,
,
z x
Z X Y
z x
X Y X Y
X Y
f z f x y dy dx
z
f x z x f x y dy dx
z
f x z x dx

c
=
c
c
| |
= +
|
c
\ .
=
} }
} }
}
Functions of Two Random Variables
One Function of Two R.V.s
Z = X + Y
If X & Y are independent,
If X & Y are independent and positive f
X
(x) and positive f
Y
(y)
( ) ( ) ( )
( ) ( )
( ) ( )
Z X Y
X Y
X Y
f z f z y f y dy
f x f z x dx
f z f z

=
=
= -
}
}
( ) ( ) ( ) ( ) ( )
0 0
z z
Z X Y X Y
f z f z y f y dy f x f z x dx = =
} }
for z>0
proof) HW
Functions of Two Random Variables
One Function of Two R.V.s
Z = X + Y
Example) Two independent exponential R.V.s with parameter
Example) Two independent uniform R.V.s in interval (0,1)
( ) ( )
x
X
f x e u x

= ( ) ( )
y
Y
f y e u y

=
( )
( )
( )( )
( )
0
2 2
0
z
z y
y
Z
z
z z
f z e e dy
e dy z e u z

=
= =
}
}
( )
0 1
2 1 2
Z
z z
f z
z z
s <

=

s <

Functions of Two Random Variables

One Function of Two R.V.s
Z = X Y
Similar to Z = X + Y case
For two independent R.V.s,
For nonnegative f
X
(x) and positive f
Y
(y)
( ) | | ( )
,
,
z y
Z X Y
F z P X Y z f x y dxdy
+

= s =
} }
( ) ( )
,
,
Z X Y
f z f z y y dy

= +
}
( ) ( ) ( ) ( ) ( )
Z X Y X Y
f z f z y f y dy f z f z

= + = -
}
( )
( )
( )
,
0
,
,
0
0
,
X Y
Z
X Y
z
f z y y dy
z
f z
z
f z y y dy

+
>

=

<
+

}
}
Functions of Two Random Variables
One Function of Two R.V.s
Z = X / Y
Find cdf
Derivative w.r.t. z
( ) | |
| | | |
, 0 , 0
Z
F z P X Y z
P X zY Y P X zY Y
= s
= s > + > <
x
y
zy
| | ( )
,
0
, 0 ,
yz
X Y
P X zY Y f x y dxdy

s > =
} }
| | ( )
0
,
, 0 ,
X Y
yz
P X zY Y f x y dxdy

> < =
} }
( ) ( ) ( )
( )
0
, ,
0
,
, ,
,
Z X Y X Y
X Y
f z yf yz y dy yf yz y dy
y f yz y dy

= +
=
} }
}
Functions of Two Random Variables
One Function of Two R.V.s
Z = X
2
+ Y
2

x
y
2
x z y =
( ) ( )
2
2 ,
,
z z y
Z X Y
z z y
F z f x y dxdy

=
} }
( )
( ) ( ) ( )
2 2
, ,
2
1
, ,
2
z
Z X Y X Y
z
f z f z y y f z y y dy
z y

= +

}
2 2
Z X Y = +
( ) ( )
2 2
2 2 ,
,
z z y
Z X Y
z z y
F z f x y dxdy

=
} }
( )
( ) ( ) ( )
2 2 2 2
, ,
2 2
, ,
z
Z X Y X Y
z
z
f z f z y y f z y y dy
z y

= +

}
Functions of Two Random Variables
One Function of Two R.V.s
Example) X & Y: independent Gaussian R.V.s with N(0,
2
)
1. Z = X
2
+ Y
2
2.
2 2
W X Y = +
( )
( )
2 2 2
2
,
2
1
,
2
x y
X Y
f x y e
o
to
+
=
( )
( )
( )
2 2 2
2 2
2
2
2
2
2 2
2
2
2 2 2
2 0 0
1 1
2
1 cos 1
cos 2
z
z y y
Z
z
z z
z
z
f z e dy
z y
e e z
dy d e u z
z
z y
o
o o
t
o
to
u
u
to to u o
+

= = =

}
} }
sin y z u =
exponential R.V. with 2
2
( )
( )
2 2
2 2
2 2
2
2
2 2
2
2
2 2
2 2 0
2 1
z
z
Z
z
z
z
z
z e
f z dy
z y
ze z
dy e u z
z y
o
o
o
to
to o

= =

}
}
Rayleigh R.V.
Functions of Two Random Variables
One Function of Two R.V.s
Example 5.44 Radius and Angle of independent Gaussian R.V.s
Zero mean and unit variance
and
Joint cdf
Joint pdf
Conclusion
R and are independent
R: Rayleigh R.V.
: Uniform R.V. in interval (0,2)
( )
1
tan Y X u

=
( )
1 2
2 2
R X Y = +
( )
( )
2
2
' / 2
2
,
0 0
'
, ' ' 1
2 2
r
r
r
R
r e
F r dr d e
u
u
u u
t t

O
= =
} }
( ) ( )
2
2
,
1
,
2
r
R
f r re u r u
t

O
=
dxdx rdrdu =
Functions of Two Random Variables
One Function of Two R.V.s
Example) nonzero mean independent Gaussian R.V.s
Nonzero m
X
, m
Y
means, unit variance
, find f
Z
(z).
HW
Hint) Substitution
Solution)
( )
1 2
2 2
Z X Y = +
sin y z u =
2 2
X Y
m m m = +
cos
X
m m | =
sin
Y
m m | =
( )
( )
( )
2 2
2
0
z m
Z
f z ze I mz
+
=
( )
( )
2
cos
cos
0
0 0
1 1
2
I e d e d
t t
q u |
q u
q u u
t t

= =
} }
Functions of Two Random Variables
One Function of Two R.V.s
Z = max(X,Y)
If X & Y are independent
X X Y
Z
Y X Y
>

=

s

x
y
y=x
(z,z)
( ) | | | |
( )
,
, ,
,
Z
X Y
F z P X z X Y P Y z X Y
F z z
= s > + s s
=
( ) ( ) ( )
Z X Y
F z F z F z =
( ) ( ) ( ) ( ) ( )
Z X Y X Y
f z f z F z F z f z = +
Functions of Two Random Variables
One Function of Two R.V.s
W= min(X,Y)
If X & Y are independent
x
y
y=x
(w,w)
Y X Y
W
X X Y
>

=

s

( ) | | | |
( ) ( ) ( )
,
, ,
,
W
X Y X Y
F w P Y w X Y P X w X Y
F w F w F w w
= s > + s s
= +
( ) ( ) ( ) ( ) ( )
W X Y X Y
F w F w F w F w F w = +
( ) ( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( )
1 1
W X Y X Y X Y
Y X X Y
f w f w f w f w F w F w f w
F w f w F w f w
= +
= +
Functions of Two Random Variables
Two Functions of Two R.V.s
Let X & Y be R.V.s with joint pdf f
X,Y
(x,y)
Given functions g(X,Y) and h(X,Y), define two new R.V.s
Find joint pdf f
Z,W
(z,w)?
Firstly, find joint cdf find joint pdf
How can find joint pdf directly??
( )
, Z g X Y =
( )
, W h X Y =
( ) ( ) ( )
( )
,
,
, , , ,
,
z
Z W
X Y
R
F z w P g X Y z h X Y w
f x y dxdy
e
( = s s

=
}}
x
( ) ( ) ( ) { }
, : ,
z
R x y g z h w = = s s x x x
Functions of Two Random Variables
Two Functions of Two R.V.s
Find joint pdf directly!
Consider g(x,y) = z and h(x,y) = w have multiple solutions (x
i
,y
i
)
Consider following probability
Need to translate above probability in terms of f
X,Y
(x,y)
| |
( ) ( )
( )
,
,
, , ,
,
Z W
P z Z z z w W w w
P z g X Y z z w h X Y w w
f z w z w
< s + A < s + A
( = < s + A < s + A

= A A
Functions of Two Random Variables
Two Functions of Two R.V.s
Find joint pdf directly!
Probability in x, y axis
Denote inverse transform
Point B'
Point C'
( ) ( ) ( )
, ,
, , ,
i X Y i i i Z W
i i
P X Y f x y f z w z w ( eA = A = A A


( )
1 1
, x g z w =
( )
1 1
, y h z w =
( ) ( )
1 1
1 1
, ,
i
g g
g z z w g z w z x z
z z
c c
+ A = + A = + A
c c
( ) ( )
1 1
1 1
, ,
i
h h
h z z w h z w z y z
z z
c c
+ A = + A = + A
c c
( ) ( )
1 1
1 1
, ,
i
g g
g z w w g z w w x w
w w
c c
+ A = + A = + A
c c
( ) ( )
1 1
1 1
, ,
i
h h
h z w w h z w w y w
w w
c c
+ A = + A = + A
c c
Functions of Two Random Variables
Two Functions of Two R.V.s
Find joint pdf directly!
Consider area
i
( )( ) ( )
( )( ) ( )( )
1 1 1 1
1 1 1 1
' ' ' ' sin
' ' cos ' 'sin ' 'sin ' ' cos
i
A B A C
A B A C A B A C
g h h g
z w z w
z w z w
g h g h
z w
z w w z
u |
| u | u
A =
=
c c c c
| || | | || |
= A A A A
| | | |
c c c c
\ .\ . \ .\ .
c c c c
| |
= A A
|
c c c c
\ .
1 1
1 1 1 1
1 1
i
g g
g h g h
z w
h h z w z w w z
z w
c c
A c c c c
c c
= =
c c A A c c c c
c c
( )
1 1
1 1
,
g g
z w
J z w
h h
z w
c c
c c
c c
c c
Jacobian
Functions of Two Random Variables
Two Functions of Two R.V.s
Find joint pdf directly!
Probability in x, y axis
Joint pdf of Z & W
( ) ( )
, ,
, ,
X Y i i i Z W
i
f x y f z w z w A = A A

( ) ( ) ( )
( )
( )
, ,
,
, , ,
1
,
,
Z W X Y i i
i
X Y i i
i
i i
f z w J z w f x y
f x y
J x y
=
=

( )
,
,
i i
i i
x x y y
g g
x y
J x y
h h
x y
= =
c c
c c
c c
c c
Functions of Two Random Variables
Two Functions of Two R.V.s
Linear Transformation
Consider following new R.V.s (where, ad-bc0)
Jacobian
Joint pdf
Z aX bY
W cX dY
= +
= +
( )
, J x y ad bc =
( ) ( ) ( )
, ,
1 1 1
, ,
Z W X Y
f z w f dz bw cz aw
ad bc ad bc ad bc
| |
= +
|

\ .
Functions of Two Random Variables
Two Functions of Two R.V.s
Example 5.45
Jacobian
Joint pdf
( )
( ) ( )
2 2 2
2 2 1
,
2
1
,
2 1
x xy y
X Y
f x y e

t
+
=

1 1
1
1 1
2
V X
W Y
( ( (
=
( ( (

1 1
1
1 1
2
X V
Y W

( ( (
=
( ( (

( )
( ) ( ) { }
2 2 2 2
, ,
2 1 2 1
2
, ,
2 2
1
2 1
V W X Y
v w
v w v w
f v w f
e

t
( (
+

+ | |
=
|
\ .
=

( )
, 1 J x y =
Pair of Jointly Gaussian R.V.s
Jointly Gaussian
R.V.s X and Y are jointly Gaussian if joint pdf has following form.
Centered at the point (m
1
,m
2
)
Bell shape depend on
1
,
2
, and
X,Y
The pdf is constant for values x and y for which the argument of the
exponent is constant:
( )
( )
2 2
1 1 2 2
,
2
1 1 2 2
,
,
2
1 2 ,
1
exp 2
2 1
,
2 1
X Y
X Y
X Y
X Y
x m x m y m y m
f x y

o o o o

to o

(
| | | || | | |

+ (
`
| | | |

( \ . \ .\ . \ .


)
=

2 2
1 1 2 2
,
1 1 2 2
2 : .
X Y
x m x m y m y m
const
o o o o
(
| | | || | | |

+ (
| | | |
(
\ . \ .\ . \ .

Pair of Jointly Gaussian R.V.s
Elliptical Contour
Depend on values of
1
,
2
, and
X,Y
When
X,Y
= 0 X and Y are independent
Contour is an elliptical with principal axes aligned with the x- and y-
axes
When
X,Y
0
Major axis of the ellipse is oriented along the angle
If variances are equal the angle is 45
, 1 2 1
2 2
1 2
2
1
arctan tan
2
X Y
o o
u
o o

| |
=
|

\ .
Pair of Jointly Gaussian R.V.s
Elliptical Contour ( )
2 2
,
, , , ,
X Y X Y X Y
m m o o
-3 -2 -1 0 1 2 3 4 5
-1
0
1
2
3
4
5
6
7
-3 -2 -1 0 1 2 3 4 5
-1
0
1
2
3
4
5
6
7
-3 -2 -1 0 1 2 3 4 5
-1
0
1
2
3
4
5
6
7
(1,3,1,3,0) (1,3,3,1,0.7)
(1,3,3,3,0.7)
-3 -2 -1 0 1 2 3 4 5
-1
0
1
2
3
4
5
6
7
(1,3,1,3,0.7)
Pair of Jointly Gaussian R.V.s
Marginal pdf
Integration to joint pdf w.r.t x or y in the interval (-,)
proof) HW
Conditional pdf
Conditional pdf is also Gaussian
( )
( )
2
2
1 1
2
1
2
x m
X
e
f x
o
to

=
( )
( )
2
2
2 2
2
2
2
y m
Y
e
f x
o
to

=
( )
( )
( )
( )
( )
( )
2
1
, 2 1
2 2
2
, 1
,
2 2
1 ,
1
exp
2 1
,
2 1
X Y
X Y
X Y
X
Y
X Y
x y m m
f x y
f x y
f y
o

o
o
to

(


`
(

)
= =

Pair of Jointly Gaussian R.V.s

Conditional Mean and Variance
Covariance
( )
1
1 , 2
2
X Y
E X Y m y m
o

o
( = +

( )
2 2
1 ,
1
X Y
VAR X Y o ( =

( ) ( )( ) ( )( )
1 2 1 2
, COV X Y E X m Y m E E X m Y m Y
(
(
( = =



( )( ) ( ) ( )
( )
( )
( ) ( ) ( )
1 2 2 1
2 1
2
1 1
2 , 2 , 2
2 2
X Y X Y
E X m Y m Y y y m E X m Y y
y m E X Y y m
y m y m y m
o o

o o
( (
= = =

( = =

= =
( ) ( )
( )
2
1
, 2
2
2
1
, 2 , 1 2
2
,
X Y
X Y X Y
COV X Y E y m
E y m
o

o
o
o o
o
(
=
(

(
= =

Pair of Jointly Gaussian R.V.s
Example 5.47 Estimation of Signal in Noise
Let Y = X + N, X and N are independent.
Find correlation coefficient between X and Y
Find x that maximizes f
X
(x|y)
( )
2
~ 0,
X
X N o
( )
2
~ 0,
N
N N o
( )
( )
, 1 2
2 2
,
1
1
X
X Y
X Y Y
N x
COV X Y
o

o o o
o o
= = =
+
( ) | | ( )
2
,
X
COV X Y E XY E X X N o ( = = + =

| | | | | | | |
0 E Y E X N E X E N = + = + =
| | ( )
2
2 2 2 2
2
X N
VAR Y E X N E X XN N o o
(
( = + = + + = +


( )
( )
( )
,
,
X Y
X
Y
f x y
f x y
f y
=
1 0 1 0
1 1 1 1
X X X X
Y N N Y
( ( ( ( ( (
= =
( ( ( ( ( (

( )
1 0
, 1
1 1
J x n = =