Chapter 4

Basic Probability
David Chow
Sep 2014

Learning Objectives
In this chapter, you will learn:
Basic probability concepts,

Conditional probability, and

Bayes Theorem.

Definitions

Probability: the chance that an event will

occur, and 0 P 1
Event: Each possible type of outcome
Simple Event: an event that can be described

by a single characteristic
Sample Space: the collection of all possible

events

Three Probability Approaches

1. a priori classical probability:
based on prior knowledge of the
process involved.
2. empirical classical probability:
based on observed data.
3. subjective probability: based on
individual judgment (which may
come from past experience, personal
opinion, or analysis of a particular
situation).
Eg: A sailor assessing the probability
of raining is an example of ___.
4

Classical Probability
Calculating classical probability
1. a priori classical probability
Probabilit y of Occurrence

X
number of ways the event can occur

T
total number of possible outcomes

2. empirical classical probability

Probabilit y of Occurrence

number of favorable outcomes observed

total number of outcomes observed

Classical Probability
Two assumptions on classical probability
1.

All outcomes are equally likely.

2.

A procedure is repeated again and again, so

that the relative frequency probability (the
formulae on the previous slide) of an event
tends to approach the actual probability.
- This is called the Law of Large Numbers
- Probability is essentially a proportion

Eg: A Priori Classical Probability

Find the probability of selecting a face card (Jack, Queen,
or King) from a standard deck of 52 cards.
Probabilit y of Face Card

X
number of face cards

T
total number of cards

X
12 face cards
3

T
52 total cards 13

Eg: Empirical Classical Probability

Find the probability of selecting a male taking statistics
from the population described in the following table:
Taking Stats

Not Taking Stats Total

Male

84

145

229

Female

76

134

210

160

279

439

Total

Probabilit y of Male Taking Stats

number of males taking stats 84

0.191
total number of people
439

Eg: Sample Space

The Sample Space is the collection of all possible
events
Eg1: All 6 faces of a die:

Eg2: All possible outcomes when having a child: Boy or Girl

Events in Sample Space

Simple event

An outcome with one characteristic

Eg: A diamond card from a deck of cards

Complement of an event A (denoted A/)

All outcomes that are not part of event A

Eg: All cards that are not diamonds

Joint event

Involves two or more characteristics simultaneously

Eg: An ace that is also red from a deck of cards

10

Visualizing Events in
Sample Space
Contingency Tables:

Ace

Not Ace Total

Black

24

26

Red

24

26

Total

48

52

Tree Diagrams:
2

Sample
Space

Full Deck
of 52 Cards

24
2
24
11

Simple vs. Joint Probability

Simple (Marginal) Probability refers to the

probability of a simple event.

Eg: P(King)

Joint Probability refers to the probability of

an occurrence of two or more events.

Eg: P(King and Spade)

12

Mutually Exclusive Events

Mutually exclusive events are events that cannot occur

together (simultaneously).
Eg1: Drawing a card
A = queen of diamonds; B = queen of clubs
Events A and B are mutually exclusive if one card is selected

Eg2: New-born baby

B = having a boy; G = having a girl
Events B and G are mutually exclusive if one child is born

13

Collectively Exhaustive Events

Collectively exhaustive events
The set of events covers the entire sample space
Hence, one of the events must occur
Eg: A deck of cards
A = aces; B = black cards; C = diamonds; D = hearts

Events A, B, C and D are collectively exhaustive (but not

mutually exclusive a selected ace may also be a heart)

Events B, C and D are collectively exhaustive and also

mutually exclusive

14

Computing Joint and

Marginal Probabilities
The probability of a joint event, A and B:

number of outcomes satisfying A and B

P( A and B)
total number of elementary outcomes
Computing a marginal (or simple) probability:

P(A) P(A and B1 ) P(A and B2 ) P(A and Bk )

Where B1, B2, , Bk are k mutually exclusive and

Eg: B = Colleges

collectively exhaustive events

15

Joint Probability Using a

Contingency Table
Event
B1

Event

B2

Total

A1

P(A1 and B1)

P(A1 and B2)

P(A1)

A2

P(A2 and B1)

P(A2 and B2)

P(A2)

Total

P(B1)

P(B2)

Joint Probabilities

Marginal (Simple) Probabilities

16

Eg: Joint Probability

P (Red and Ace)

number of cards that are red and ace 2

total number of cards

52
Ace

Not
Ace

Total

Black

24

26

Red

24

26

Total

48

52

17

Eg: Marginal (Simple) Probability

P(Ace)
P( Ace and Re d) P( Ace and Black)

Ace

Not Ace

2
2
4

52 52 52

Total

Black

24

26

Red

24

26

Total

48

52

18

Probability Summary So Far

What does P (head) = mean?

There must be a head out of the next 2 tosses.

The result must be head or no head, so P (head) must be
either 1 or 0.
True or False?

Probability measures the likelihood that an event will occur.

0 P(A) 1 for any event A.

The sum of the probabilities of all mutually exclusive and

collectively exhaustive events is 1.
P(A) + P(B) + P(C) = 1
A, B, and C are mutually exclusive and collectively
exhaustive

Certain

0.5

Impossible

Marginal probability = sum of joint probabilities if ____

19

General Addition Rule

General Addition Rule:

P(A or B) = P(A) + P(B) - P(A and B)
If A and B are mutually exclusive, then
P(A and B) = 0, so the rule can be simplified:

P(A or B) = P(A) + P(B)

for mutually exclusive events A and B
20

Eg: General Addition Rule

Find the probability of selecting a male or a statistics student
from the population described in the following table:
Taking Stats

Not Taking Stats

Total

Male

84

145

229

Female

76

134

210

160

279

439

Total

P(Male or Stat) = P(M) + P(S) P(M or S)

= 229/439 + 160/439 84/439 = 305/439
21

Eg: Titanic Mortality

Men

Women

Boys

Girls

Total

Survived

332

318

29

27

706

Died

1360

104

35

18

1517

Total

1692

422

64

45

2223

1. P (a man or a boy) =
2. P (a man or a survivor) =
22

Conditional Probability

23

Case Study: Genius & Maniac

Aristotle: There is no great genius

without a mixture of madness

Scientists at Kings College

London found that
Top graders at school were

four times more likely to

develop bipolar disorder
(manic depression) than
average students
The link was strongest among

those who studied music or

literature

24

Conditional Probability
A conditional probability is the probability of one event,

given that another event has occurred:

P(A and B)
P(A | B)
P(B)

The conditional
probability of A given
that B has occurred

P(A and B)
P(B | A)
P(A)

The conditional
probability of B given
that A has occurred

where P(A and B) = joint probability of A and B,

P(A) = marginal probability of A, and P(B) = marginal probability of B

The Venn diagram provides an intuitive explanation.

25

Eg: Conditional Probability

Of the cars on a used car lot, 70% have air

conditioning (AC) and 40% have a CD player

(CD). 20% of the cars have both.
What is the probability that a car has a CD player,

given that it has AC ?

I.e., find P(CD | AC).

26

Eg: Conditional Probability

CD

No CD

Total

AC

0.2

0.5

0.7

No
AC

0.2

0.1

0.3

Total

0.4

0.6

1.0

P(CD and AC) .2

P(CD | AC)
.2857
P(AC)
.7
Given AC, we only consider the top row (70% of the cars). Of these,
20% have a CD player. 20% of 70% is about 28.57%.
27

Eg: Conditional Probability in

Decision Trees
Given CD or
no CD:

All
Cars

.2
.4

.2
.4
.5
.6

.1
.6

P(CD and AC) = .2

P(CD and AC/) = .2

P(CD/ and AC) = .5

P(CD/ and AC/) = .1

28

Eg: Conditional Probability in

Decision Trees
.2
.7

Given AC or
no AC:

All
Cars

.5
.7
.2
.3

.1
.3

P(AC and CD) = .2

P(AC and CD/) = .5

P(AC/ and CD) = .2

P(AC/ and CD/) = .1

29

Eg: Card Counting

What is the probability of

getting a high-valued card?

David Ho ()

David Ho is a famous researcher of AIDS for the

AIDS cocktail treatment

Movie: 21

Inspired by the true story of the MIT Blackjack Team,

which is split into two groups:

1.

"Spotters" play the minimum bet and do the counting.

They send secret signals to the "big players

2. Big players place large bets when the count is

favorable
30

Statistical Independence
Two events are independent if and only if:

P(A | B) P(A)
Events A and B are independent when the

probability of one event is not affected by the

other event.

31

Eg: Titanic Mortality Again

Men

Women

Boys

Girls

Total

Survived

332

318

29

27

706

Died

1360

104

35

18

1517

Total

1692

422

64

45

2223

Probability of getting a women or child if a survivor

is randomly selected
P(man | died) =
Are the events man and died statistically

independent?
32

Eg1: Statistical Independence

Given the following contingency table:
B

10

30

25

35

What is the probability of

(a) A|B ?
(b) A|B?
(c) A|B ?

(d) Are events A and B statistically independent?

33

Multiplication Rule
Multiplication rule for two events A and B:

P(A and B) P(A | B) P(B)

If A and B are independent, then P(A | B) P(A)

and the multiplication rule simplifies to:

P(A and B) P(A) P(B)

34

Multiplication Rule
P (A and B) is the product of the probability

of event A and the probability of event B.

But the probability of B has to take into

account the previous occurrence of event A.

Hence the conditional probability P(B|A)

appears in the multiplication rule.

35

Multiplication Rule
Eg: Find P (2 and 2) if a die is rolled twice
Eg: Find P (2 Aces) if two cards are drawn

from a deck
with replacement or
without replacement

36

Multiplication Rule
Statistical independence simplifies the multiplication

rule, but sampling without replacement implies

dependence.
General rule: Assume stat. independence when

sample size (n) is within 5% of the population (N),

i.e., n =< 0.05 (N)
Eg: A sample of 12 camera are drawn from a population

of 1,000.
Given a 5% defect rate, find P (all 12 cameras are good).

37

Eg: Multiplication Rule

Suppose a city council is composed of 5

democrats, 4 republicans, and 3 independents.

Find the probability of randomly selecting a

democrat followed by an independent.

P(I and D) P(I | D) P(D) (3/11)(5/12) 5/44 .114

Note that after the democrat is selected (out of 12

people), there are only 11 people left in the

sample space.

38

Marginal Probability Using

Multiplication Rules
Marginal probability for event A:
P(A) P(A | B1 ) P(B1 ) P(A | B2 ) P(B2 ) P(A | Bk ) P(Bk )

Where B1, B2, , Bk are k mutually exclusive

and collectively exhaustive events

39

Bayes Theorem
Bayes Theorem was developed by Thomas

Bayes in the 18th Century.

It is used to revise previously calculated

probabilities based on new information.

It is an extension of conditional probability.

40

Bayes Theorem
P(A | Bi )P(Bi )
P(Bi | A)
P(A | B1 )P(B1 ) P(A | B2 )P(B2 ) P(A | Bk )P(Bk )
where:
Bi = ith event of k mutually exclusive
and collectively exhaustive events

A = new event that might impact P(Bi)

41

Eg: Bayes Theorem

A drilling company has estimated a 40% chance of striking

oil for their new well. A detailed test has been scheduled for
more information.
Historically,
60% of successful wells have had detailed tests, and
20% of unsuccessful wells have had detailed tests.
Given that this well has been scheduled for a detailed test,

what is the probability that the well will be successful?

42

Eg: Bayes Theorem

Let S = successful well, U = unsuccessful well
P(S) = .4 , P(U) = .6

(prior probabilities)

Define the detailed test event as D

Conditional probabilities:

P(D|S) = .6

P(D|U) = .2

Find P(S|D)

43

Eg: Bayes Theorem

Apply Bayes Theorem:
P(D | S)P(S)
P(D | S)P(S) P(D | U)P(U)
(.6)(.4)

(.6)(.4) (.2)(.6)
.24

.667
.24 .12

P(S | D)

So, the revised probability of success, given that this

well has been scheduled for a detailed test, is .667
44

Eg: Bayes Theorem

Given the detailed test, the revised probability of a

successful well has risen to .667 from the original estimate

of 0.4.
Joint

Revised

Prob.

Conditional
Prob.

Prob.

Prob.

S (successful)

.4

.6

.4*.6 = .24

.24/.36 = .667

U (unsuccessful)

.6

.2

.6*.2 = .12

.12/.36 = .333

Event

Prior

45

Review

46

Eg2: Addition Rule

Actually purchased
Planned to purchase

Yes

No

Total

Yes

200

50

250

No

100

650

750

Total

300

700

1000

P (planned to purchase) =
P (planned or actually purchased) =
Use the contingency table in this example to

illustrate the addition rule.

47

Eg3: Statistical Independence

A sample of 600 respondents was selected in Beijing to study

consumer behavior, with the following results:

Enjoys Shopping
For Clothing

Gender
Male

Female

Total

Yes

163

269

432

No

125

43

168

Total

288

312

600

48

Eg3: Continued
(a) Suppose the respondent chosen is a female.

What is the probability that she does not enjoy

shopping for clothing?
(b) Suppose the respondent chosen enjoys shopping

for clothing. What is the probability that the

individual is a male?
(c) Are enjoying shopping for clothing and the

gender of the individual statistically independent?

Explain.
49

Eg4: Bayes Theorem

1. If P(B) = 0.05, P(A | B) = 0.80, P(B) = 0.95, and P(A | B) = 0.40, find
P(B | A).
2. An advertising executive is studying television viewing habits of
married men and women during prime-time hours.
Based on past viewing records, the executive has determined that
during prime time, husbands are watching television 60% of the time.
When the husband is watching television, 40% of the time the wife is
also watching. When the husband is not watching television, 30% of
the time the wife is watching television.
Find the probability that
(a) If the wife is watching television, the husband is also watching
television.
(b) The wife is watching television in prime time.

50

Video

51

1969

LUM



o

LEAM
HAM BAN LAN

MOU

http://hk.youtube.com/watch?v=eN68W2fWwR0&feature=related

52

2011-05-08

http://www.youtube.com/watch?v=Np4adVu-om0

53