Académique Documents
Professionnel Documents
Culture Documents
Berlin Chen
Department of Computer Science & Information Engineering
National Taiwan Normal University
Reference:
- D. P. Bertsekas, J. N. Tsitsiklis, Introduction to Probability , Sections 3.4-3.6
Multiple Continuous Random Variables (1/2)
• Marginal Probability
P X A P X A and Y ,
X A f X ,Y x , y dydx
– We have already defined that
P X A f X x dx
X A
f X x f X ,Y x , y dy
Similarly
f Y y f X ,Y x , y dx
Probability-Berlin Chen 3
An Illustrative Example
• Example 3.10. Two-Dimensional Uniform PDF. We are told that
the joint PDF of the random variables X and Y is a constant c
on an area S and is zero outside. Find the value of c and the
marginal PDFs of X and Y .
The correspond ing uniform joint PDF on
an area S is defined to be (cf. Example 3.9)
1
, if x,y S
f X ,Y x,y Size of area S
0, otherwise
1
f X ,Y x,y for x,y S
4
for 1 x 2 for 1 y 2
f X x 14 f X,Y x,y dy f Y y 12 f X,Y x,y dx
1 3 1 1
14 dy 12 dx for 3 y 4
4 4 4 4
for 2 x 3 for 2 y 3 f Y y 12 f X,Y x,y dx
f X x 23 f X,Y x,y dy f Y y 13 f X,Y x,y dx 1
12 dx
1
1 1 1 1 4 4
23 dy 13 dx
4 4 4 2 Probability-Berlin Chen 4
Joint CDFs
• If X and Y are two (either continuous or discrete)
random variables associated with the same experiment ,
their joint cumulative distribution function (Joint CDF) is
defined by
F X ,Y x , y P X x , Y y
– If X and Y further have a joint PDF f X ,Y ( X and Y are
continuous random variables) , then
F X ,Y x , y P X x , Y y xy , for 0 x,y 1
Y
0,1 1,1
X
0,0 1,0
2 FX ,Y x, y
1 f X ,Y x, y , for all x, y in the unit square
xy
Probability-Berlin Chen 6
Expectation of a Function of Random Variables
E Z E aX bY a E X b E Y
Z
We will see in Section 4.1 methods for computing the PDF of (if it has one). Probability-Berlin Chen 7
More than Two Random Variables
P X B A B f X A x dx
• Normalization property
fX A x dx 1
Probability-Berlin Chen 9
Conditioning PDF Given an Event (2/3)
– If A can be described in terms of X ( A is a subset of the real
line with P X A 0 ), the conditional PDF is defined as a
nonnegative function f X A x satisfying
f X x
, if x A
fX A x P X A
0, otherwise
• The conditional PDF is zero outside the
conditioning event
fX A remains the same shape as
and for any subset B f X except that it is scaled along
P X B, X A
the vertical axis
P X B X A
P X A
A B f X x dx
P X A
A B f X A x dx
– Normalization Property f X A x dx A f X A x dx 1
Probability-Berlin Chen 10
Conditioning PDF Given an Event (3/3)
P X x P Ai P X x Ai
n and use the total probability theorem
from Chapter 1
i 1
n
x
f X t dt P Ai x f X Ai t dt
i 1
Taking the derivative of both sides with respect to x
n
f X x P Ai f X Ai x
i 1 Probability-Berlin Chen 11
Illustrative Examples (1/2)
1 1 3 1 1
For 0 y 5, PY y
4 5 4 15 10
Total Probability theorem: 1 3 1 1
For 5 y 15, PY y 0
PY y P A PY A y P B PY B y 4 4 15 20
Probability-Berlin Chen 13
Conditioning one Random Variable on Another
f X ,Y x ,3 .5 1/ 4
f X Y x 3 . 5 1
f Y 3 .5 1/ 4
f X ,Y x , 2 .5
f X Y x 3 . 5
1/ 4
1/ 2
f Y 2 .5 1/ 2
f X ,Y x ,1 .5
f X Y x 3 .5
1/ 4
1
f Y 1 .5 1/ 4
πr
(Notice here that PDF f Y y is not uniform) For each value y , f X Y x y is uniform
Probability-Berlin Chen 16
Conditional Expectation Given an Event
i 1
• Where are disjoint events with P Ai 0 for
A1 , A2 , , An
each i , that form a partition of the sample space
Probability-Berlin Chen 17
An Illustrative Example
• Example 3.17. Mean and Variance of a Piecewise Constant PDF.
Suppose that the random variable X has the piecewise constant
PDF 1 / 3, if 0 x 1,
f X x 2 / 3, if 1 x 2 ,
0, otherwise.
Define event A1 X lies in the first interval [0,1]
event A2 X lies in the second interval [1,2]
P A1 01 1 / 3 dx 1 / 3, P A2 12 2 / 3 dx 2 / 3
f X x f X x
1, 0 x 1 1, 1 x 2
fX A1 x P X A 1 fX A2 x P X A 2
0, otherwise 0, otherwise
E X Y y xf X Y x y dx
E g X Y y g x f X Y x y dx
E g X , Y Y y g x , y f X Y x y dx
Probability-Berlin Chen 19
Total Probability/Expectation Theorems
P A
P A Y y f Y y dy
E X E X Y y f Y y dy
E g X E g X Y y f Y y dy
E g X , Y E g X , Y Y y f Y y dy
Probability-Berlin Chen 20
Independence
f X ,Y x , y f Y y f X Y x y f X x f Y X
y x
• We therefore have
• Or
fY X y x fY y , for all y and all x with f X x 0
Probability-Berlin Chen 21
More Factors about Independence (1/2)
• If two continuous random variables X and Y are
independent, then
– Any two events of the forms X Aand Y B are
independent
P X A , Y B x A y B f X ,Y x , y dydx
x A y B f X x f Y y dydx
x A f X x dx y B f Y y dy
P X A P Y B
– It also implies that
F X ,Y x , y P X x , Y y P X x P Y y F X x FY x
– The converse statement is also true (See the end-of-chapter
problem 28)
Probability-Berlin Chen 22
More Factors about Independence (2/2)
• Therefore,
E g X h Y E g X E h Y
Probability-Berlin Chen 23
Recall: the Discrete Bayes’ Rule
P Ai B
P Ai P B Ai
P B Multiplication rule
P Ai P B Ai Total probability theorem
nk 1 P Ak P B Ak
P Ai P B Ai
P A1 P B A1 P An P B An
Probability-Berlin Chen 24
Inference and the Continuous Bayes’ Rule
X Y
Measurement Inference
f X x fY X y x f X Y x y
f X ,Y x , y f X x fYy x
f X Y x y
X
fY y f X t f Y X y t dt
Note that
f X fY X
f X ,Y f Y f X Y
Probability-Berlin Chen 25
Inference and the Continuous Bayes’ Rule (2/2)
Inference about a Discrete Random Variable
• If the unobserved phenomenon is inherently discrete
– Let N is a discrete random variable of the form N n that
represents the different discrete probabilities for the unobserved
phenomenon of interest, and p N be the PMF of N
P N n Y y P N n y Y y
P N n P y Y y N n
P y Y y
p N n f Y N
y n
f Y y
p N n f Y y n
N
Total probability theorem
p N i f y i
Y N
i Probability-Berlin Chen 26
Illustrative Examples (1/2)
• Example 3.19. A lightbulb produced by the General Illumination
Company is known to have an exponentially distributed lifetime Y .
However, the company has been experiencing quality control
problems. On any given day, the parameter of the PDF of Y
is actually a random variable, uniformly distributed in the interval
1, 3 / 2 .
– If we test a lightbulb and record its lifetime ( Y y ), what can
we say about the underlying parameter ?
f Y y e y , y 0, 0 Conditioned on , Y has a exponential distribution
with parameter
2, for 1 3 / 2
f
0, otherwise
f f Y y 2 e y
f Y y , for 1 λ 3 / 2
3/ 2
1 f t f Y y t dt 3/ 2 ty
1 2te dt
Probability-Berlin Chen 27
Illustrative Examples (2/2)
p S 1 f Y y 1 p S 1 f Y y 1
P S 1 Y y
S S
fY y p S 1 f Y S y 1 p S 1 fY S y 1
1
e y 1 / 2
2
p
2
1 1
e y 1 / 2 1 p e y 1 / 2
2 2
p
2 2
e y
2
1 / 2
pe y pe y
e
y 2 1 / 2 y
pe e 1 p e y
y 2 1 / 2 pe y 1 p e y
Probability-Berlin Chen 28
Inference Based on a Discrete Random Variable
f Y y P A Y y
fY y
A
P A
f Y y P A Y y
?
f Y t P A Y t dt
P A fY A y f Y y P A Y y
P A f Y A y dy
f Y y P A Y y dy
Probability-Berlin Chen 29
Recitation
Probability-Berlin Chen 30