Bayes Theorem

STAT 400
August 30, 2016
Definition: Law of total probability. Let the sample space be partitioned into m mutually exclusive
portions, S = m
i=1 Bi such that Bi Bj = i 6= j.
Event A is defined as, A = m
i=1 (Bi A).
Accordingly, P (A) is,
P (A) =

m
X

P (Bi A) =

i=1

m
X

P (Bi ) P (A|Bi )

i=1

Definition Bayes Theorem

P (Bk |A) =

P (Bk ) P (A|Bk )
P (Bk A)
= Pm
P (A)
i=1 P (Bi ) P (A|Bi )

Example 1 There are three boxes, each with one drawer on each of two sides. Each drawer contains
a coin. One box has a gold coin on each side (GG), one a silver coin on each side (SS), and the
other a gold coin on one side and a silver coin on the other (GS). A box is chosen at random, a
random drawer is opened, and a gold coin is found inside it.
a. What is the probability you see a gold coin?
P (see G) = P (GS)P (see G|GS) + P (SS)P (see G|SS) + P (GG)P (see G|GG)
=

11
1
1
+0+ =
32
3
2

b. What is the probability the other coin is gold given you see a gold coin?
P (G|see G) =

P (GG)P (see G|GG)

2
=
P (see G)
3

STAT 400

Bayes Theorem

Example 2 Suppose we roll three fair 6-sided dice.

a. What is the probability that the dice sum to 12? (Hint: Let Bi represent the first die)
We can use the total probability formula. Let A be the event that the sum of the three
dice is 12 and Bi is the value of the first die.
Sum of 2nd & 3rd Dice
1st Die 2

10

11

12

2
3

2
3

5
6

5
6

P (A) =

6
X

P (Bi A) =

6
X

P (Bi ) P (A|Bi )

i=1

i=1

6
1X
1 5+6+5+4+3+2
P (A|Bi ) =
6 i=1
6
36

52
0.116
63

b. Compute the conditional probability that the first dice shows 4, given that the sum of the
three numbers showing is 12.
We want P (Bi = 4|A), which is
P (Bi = 4|A) =
=

P (Bi = 4 A)
P (Bi = 4) P (A|Bi = 4)
=
P (A)
P (A)
1
6

5
36

52
63

1
5

Stepanov, Culpepper

STAT 400

Bayes Theorem

Example 3 In a certain population, the proportion of individuals who have a particular disease is
0.025. A test for the disease is positive in 94% of the people who have the disease and in 4% of the
people who do not. Let + indicate a positive test and D the presence of the disease.
a. Find the probability of receiving a positive reaction from this test, i.e., P (+).
We can use the total probability probability formula. First, note that P (D) = 0.025,
P (+|D) = 0.94, and P (+|Dc ) = 0.04. The law of total probability implies that,

P (+) = P (D) P (+|D) + P (Dc ) P (+|Dc )

47
39
1
+
40
50
40
  

1
1
125
=
=
40
25
2





1
25
1
= 0.0625
16


(1)
(2)
(3)

b. If a person received a positive reaction from this test, what is the probability that he/she has
the disease?
We want P (D|+), which can be found using Bayes theorem,

P (D|+) =

P (+|D) P (D)
P (+)


47
50



1
40

1
16

47
= 0.376
125

c. If a person received a negative reaction from this test, what is the probability that he/she
doesnt have the disease?
We want P (Dc |+c ), which can also be found using Bayes theorem,

P (Dc |+c ) =

P (+c |Dc ) P (Dc )

P (+c )


96
100



39
40

15
16

48 (13)
= 0.9984
252
Stepanov, Culpepper

STAT 400

Bayes Theorem

Note that for this problem we could have also filled in the following table,
+

+c

0.0235

0.0015

0.0250

Dc

0.0390

0.9360

0.9750

Total

0.0625

0.9375

1.000

Example 4 A warehouse receives widgets from three different manufacturers, A (50%), B (30%),
and C (20%). Suppose that 2% of the widgets coming from A are defective, as are 4% of the
widgets coming from B, and 7% coming from C.
a. Find the probability that a widget selected at random at this warehouse is defective (D).
We can use the total probability probability formula.

P (D) = P (A) P (D|A) + P (B) P (D|B) + P (C) P (D|C)

1
1
=
+
2
50

1 1
3
=
+
+
25 4 10
 

3
1
1
7
+
10
25
5
100

 
7
1 9
=
= 0.036
20
25 10


 

b. Suppose a widget that was selected at random is found to be defective. What is the probability
that it came from manufacturer A? Manufacturer B? Manufacturer C?
P (D|A) P (A)
=
P (D)

P (D|B) P (B)
P (B|D) =
=
P (D)

P (D|C) P (C)
P (C|D) =
=
P (D)

P (A|D) =

 

1
1
50
2
 
1
9
25 10
1
25
1
25



3
10
 
9
10

5
18

1
3

7
18

 

7
1
100
5
 
9
1
25 10

Stepanov, Culpepper

STAT 400

Bayes Theorem

Example 5 The game of craps is played by rolling two fair, 6-sided dice. On the first roll, if the
sum of the two numbers showing equals 2, 3 or 12 then the player immediately loses. If the sum
equals 7 or 11, the player immediately wins. If the sum equals any other value, then this number
becomes the players point. The player then repeatedly rolls two dice until such time as he or she
rolls the point value (in which case he or she wins) or rolls a 7 (in which case he or she loses).
a. Suppose the players point is equal to 4. Conditional on this, what is the probability that he
or she will win (i.e. roll another 4 before rolling a 7)? (Hint: the final roll will either be a 4
or a 7; what is the conditional probability it is a 4?).
We need to add the probabilities of winning on the 1st, 2nd, 3rd, . . . rolls. Note that
DeMorgans Law implies that (4 7)c = 4c 7c . We end up with an infinite sum which
you should recognize as the geometric series,

P (win|4) = P (win on 1st roll|4) + P (win on 2nd roll|4) + P (win on 3rd roll|4) +
= P (4) + P (4c 7c ) P (4) + P (4c 7c ) P (4c 7c ) P (4) +

27
3 X
= P (4)
[P (4 7 )] =
36 i=0 36
i=0

i

1
3

There is another way to think about this problem. That is, the game ends when either
a 4 or 7 occurs. In this case, winning is defined as rolling a four given the game ends.
So, we can see that,
P (4 (4 7))
P ((4 4) (4 7))
=
P (4 7)
P (4) + P (7)
P (4 4) + P (4 7))
P (4)
1
=
=
=
P (4) + P (7)
P (4) + P (7)
3

P (win|4) = P (4|4 7) =

Stepanov, Culpepper

STAT 400

Bayes Theorem

b. For 2 i 12 let pi be the conditional probability that the player will win, conditional on
having rolled i on the first roll. Compute pi for all 2 i 12. (Hint: youve already done
this for i = 4 in part (a). The cases = 2, 3, 7, 11, 12 are trivial.)
Derivation

pi
P (win|2)

P (win|3)

P (win|4)

1
3

P (win|5)

4
36

P (win|6)

5
36

P h 26 ii

2
5

5
11

P (win|8)

= P (win|6) =

5
11

P (win|9)

= P (win|5) =

2
5

P (win|10)

= P (win|4) =

1
3

i=0

36

P h 25 ii
i=0

P (win|7)

36

P (win|11)

P (win|12)

c. Compute the overall probability that a player will win at craps. (Hint: Use part (b) ).
Note that we can compute P (win) by using the law of total probability,
P (win) =

12
X

P (k) P (win|k)

k=2

2
3
1
4
2
5
5
6
1
(0) +
(0) +
+
+
+
(1) +
=
36
36
36
3
36
5
36
11
36
          
 
5
5
4
2
3
1
2
1
+
+
+
(1) +
(0)
36
11
36
5
36
3
36
36


1
8 25
=
1+ +
+4
18
5 11
 
1 1
244
=
[55 + 88 + 125 + 220] =
0.4929
18 55
495


 

 



Stepanov, Culpepper