STAT2001 Tutorial 2

Sept. 23, 2014

1. Law of Total Probability and Bayes Theorem

(1) Law of Total Probability
Let B1 , ..., Bn be disjoint events that form a partition of the sample space S
and assume that P (Bi ) > 0, for all i. And
B1 [ B2 [ ... [ Bn = S
For any event A, we have
P (A) = P (S \ A)

= P ((B1 [ B2 [ ... [ Bn ) \ A)

= P ((B1 \ A) [ ... [ (Bn \ A))

= P (B1 \ A) + ... + P (Bn \ A)

= P (B1 )P (A|B1 ) + ... + P (Bn )P (A|Bn )

(2) Bayes Theorem
If P (A) > 0, then
P (Bk |A) =

P (A \ Bk )
P (Bk )P (A|Bk )
= Pn
.
P (A)
i=1 P (Bi )P (A|Bi )

Bayes Theorem is a common and useful technique for inference.

Without any ancillary information, the (prior) probability that event Bk
occurs is P (Bk ). Now suppose we observe event A happens, and know events
B1 , ..., Bn have certain association relationship with event A. We represent this
knowledge in terms of conditional probability P (A|B1 ), ..., P (A|Bn ). By using
Bayes Theorem, the (posterior) probability that Bk occurs is updated to be
P (Bk |A).
Briefly speaking, Bayes Theorem provides us a framework to unify the conditional information together to draw statistical inference.

Exercise 1 There is a copycat AK47 for shooting test. During the test, we
will have three shoot respectively. The probability that hit the target in first
time is 0.4, that hit the target in second time is 0.5 and that hit the target in
third time is 0.7. If the target is hit one time, two time and three time, the
probability of destroy is 20%, 60% and 80% respectively. What is the probability that target destroy?
Exercise 2 Suppose that 5% of men and 0.25% of women are color-blind.A
person is chosen at random and that person is color-blind. What is the probability that the person is male?(Assume males and females to be in equal numbers.)
Exercise 3 A computer can send any one of three characters A, B, C to
a printer. During each transmission, each character has equal chance to be
sent. However, the printer may not print the same character as transmitted
for transmission errors. The probability that an A will be incorrectly printed
as B is 0.2, and that of C to be printed as A is 0.5. Other printer errors are
impossible.
1. Suppose two characters are transmitted independently. What is the probability that two As will be printed?
2. Suppose that a single character is transmitted. If B is printed, what is the
probability that a B was transmitted?
2. Discrete random variables and Probability Mass Function(p.m.f.)
1. A random variable(r.v.) is a function X that maps each element s 2 S
to a real number x, that is X(s) = x.
Discrete random variable:If the set {x : X(s) = x, s 2 S} has finite or
countably infinite real numbers.
Remark: A random variable X is a function from sample space S to R.
X itself is not an event but X = some real number is.
X=1
X=4

X=2

Here, X is a r.v. valued on four

numbers, thus {X = i}s are
four events contained in S !

X=3
the whole space S

2. The Probability Mass Function (p.m.f.) of a discrete random variable X is a function f(x) satisfies
(1) fP(x) > 0, 8x 2 S;
(2) x2S f (x) =
P1.
(3) p(x 2 A) = x2A f (x), 8A S;
Exercise 4 For each of the following discrete random variable X, determine
the constant c such that f (x) is the p.m.f. of X.
1. f (x) = xc , x = 1, 2, ..., n.
2. f (x) = c

1 x
4

, x = 1, 2, 3, ....

3. Functions of Discrete Random Variable

For any real function g : Y = g(X), Y also a discrete random variable.
Moreover, the p.m.f. of Y is completely determined by the p.m.f. of X.
Note: Y (s) = g X(s). For (g X)(s) is a real function of s, Y is a random
variable.
4. Cumulative Distribution Function(c.d.f.)
Definition : F (x) = P (X x).
Note: both p.m.f and c.d.f are two equivalent ways to describe a discrete
random variable for we could derive the p.m.f. of a r.v. from its c.d.f. and vice
versa.
Exercise 5
(I)Roll a fair dice twice independently. Let X = 0 if the two numbers we get
are the same, X = 1 if the first number is smaller than the second number,
and X = 1 otherwise. Find both the p.m.f and c.d.f of X and draw graphs for
them.
(II)Set Y = |X|, find the p.m.f of Y and give an interpretation of Y being a
random variable.
5. Mathematical Expectation
The Expected value or mean of a random variable is a measure of the middle
part of the random variables distribution. For a discrete random variable X
with p.m.f. f (x), the expected value of X is defined as:
X
X
E(X) =
xf (x) =
xP (X = x).
x2S

x2S

provided that the sum exists.

If u(X) is a function of X, the mathematical expectation of u(X) is

X
X
E[u(X)] =
u(x)f (x) =
u(x)P (X = x).
x2S

x2S

Properties of Expectation
1. If c is a constant, E(c) = c.
2. If c1 and c2 are two constants, X and Y are two random variables,
E[c1 X + c2 Y ] = c1 EX + c2 EY.
3. If X and Y are independent, E(XY ) = E(X)E(Y ).
Exercise 6 Suppose the largest prize of Mark Six lottery is 1500 HKD, and
each bet costs 10 HKD. How small the probability is when the expectation income is below zero(which means we should not buy it)?
5. Variance
To measure the variation or dispersion of the distribution of the random variable
X (about its own mean), we are going to introduce Variance.
V ar(X) = E[X

E(X)]2 = E[X 2 ]

[EX]2 .

A more interpretable measure of variation (which is in the same unit as X) is

the standard deviation of X,
p
SD(X) = V ar(X).
Properties of Variance
1. If c is a constant, Var(c) = 0;
2. If c is a constant and X is a random variable, Var(cX) = c2 V ar(X);
3. If X and Y are independent, V ar(X + Y ) = V ar(X) + V ar(Y ).

Sample mean, variance and standard deviation

If x1 , x2 , , xn are n independent observations, the sample mean, variance and
standard deviation are defined
Pn as follows:
1. Sample mean=
x = n1 i=1 x
i;
P
n
2. Sample variance=s2 = n 1 1 i=1 (xi x
)2 ;
3. Sample standard deviation=s