Introduction to Probability

Dr. T. Landolsi
Lecture Objectives

▪ Random experiments
▪ Counting techniques
▪ Probability axioms and rules

T. Landolsi - Stochastic Systems 2

Random Experiments
Random Experiments and Probability

▪ Probability: mathematical study of random phenomena 𝒮

▪ Random experiment: procedure (may be hypothetical) producing 𝐴 𝐵

indeterministic result
𝑒2
𝑒1
▪ example: experiment consisting of tossing fair coin 5 times

▪ Outcome is a possible result of a simple experiment 𝐶

▪ outcome. getting H after tossing coin once
𝑒𝑘

▪ Event is a set or a collection of outcomes

▪ event. getting HHHTT after tossing coin 5 times
Venn Diagram
▪ Sample space: set of all possible outcomes

T. Landolsi - Stochastic Systems 4

Probability Calculation

Empirical Method Classical Method

▪ Consider random experiment: ▪ Experiment with equally likely fundamental

outcomes
▪ experiment repeated 𝑛 times
▪ 𝑛𝐸 , count number of ways event 𝐸 can occur
▪ event 𝐸 occurs 𝑛𝐸 times
▪ 𝑛𝑠 , count number of all possible outcomes
▪ Relative frequency of event 𝐸:
𝑓𝐸 = 𝑛𝐸 /𝑛 ▪ uses counting techniques

▪ Probability, 𝑝𝐸 , of event 𝐸 as 𝑛 → ∞:
𝑛𝐸 ▪ Probability, 𝑝𝐸 , of event 𝐸 :
𝑝𝐸 ≈ 𝑓𝐸 =
𝑛
# of ways 𝐸 can occur 𝑛𝐸
𝑝𝐸 = =
# of possible outcomes 𝑛𝑆

T. Landolsi - Stochastic Systems 5

Examples

▪ Random experiment: Roll six-sided die ▪ Event 𝐵: get a six

▪ Outcomes: 1, 2, 3, 4, 5, or 6 ▪ 𝐵 = {6}, 𝑛𝐵 = 1, 𝑝𝐵 = 1/6

▪ Sample space: 𝑆 = {1,2,3,4,5,6}

▪ Event 𝐶: get number > 13
▪ 𝐶 = { }, 𝑛𝐶 = 0, 𝑝𝐶 = 0
▪ Event 𝐴: get even number
▪ 𝐶 is called impossible event
▪ 𝐴 = {2,4,6}, and 𝑛𝐴 = 3
𝑛𝐴 3
𝑝𝐴 = = = 1/2 ▪ Event 𝐷: get number > 0
𝑛 6
▪ 𝐷 = {1,2,3,4,5,6}, 𝑛𝐷 = 6, 𝑝𝐷 = 1
▪ 𝐷 is called certain event

T. Landolsi - Stochastic Systems 6

Counting Techniques
Counting Techniques

▪ Calculating probabilities may involve counting techniques:

▪ There are three main counting techniques:

▪ using multiplication rule
▪ using permutations
▪ using combinations

T. Landolsi - Stochastic Systems 8

Counting using Multiplication Rule

▪ If a task consists of a sequence of choices in which there are 𝑝 selections for the first choice, 𝑞
selections for the second choice, 𝑟 selections for the third choice, and so on, then the task of
making these selections can be done in as many ways as:
𝑝×𝑞×𝑟×⋯

▪ Example. A computer engineer studies impact of three parameters 𝑝1 , 𝑝2 , 𝑝3 on performance of a

database server. Parameter 𝑝1 can take 5 values, 𝑝2 can take 2 values, and 𝑝3 can take 4 values.
In how many ways can engineer design an experiment by changing these parameters values?

▪ Solution. Using the multiplication rule, there are as many ways to do this experiment as:
𝑝1 × 𝑝2 × 𝑝3 = 5 × 2 × 4 = 40

9
Counting using Permutations

▪ Permutation is ordered arrangement in which 𝑟 objects are chosen from 𝑛 distinct objects without repetition

▪ The symbol 𝑛𝑃𝑟 represents the number of permutations of 𝑟 objects selected from 𝑛 objects:
𝑛!
𝑃
𝑛 𝑟 =
𝑛−𝑟 !
▪ where 𝑛!, called “𝑛 factorial”, is given by: 𝑛! = 𝑛 × 𝑛 − 1 × ⋯ × 2 × 1

▪ 𝑛𝑃𝑟 is used to determine the number of arrangements of 𝑟 objects chosen from 𝑛 objects, in which:
▪ 𝑛 objects are distinct

▪ repetition of objects is not allowed

▪ order is important

10
Counting using Combinations

▪ A combination is a collection, without regard to order, of 𝑛 distinct objects without repetition

𝑛
▪ The symbol 𝑛𝐶𝑟 , also noted as and pronounced “𝑛 choose 𝑟” represents the number of combinations of 𝑛 distinct
𝑟
objects taken 𝑟 at a time:
𝑛!
𝑛𝐶𝑟 =
𝑛 − 𝑟 ! 𝑟!

▪ 𝑛𝐶𝑟 is used to determine the number of combinations of 𝑟 objects chosen from 𝑛 objects, in which:
▪ 𝑛 objects are distinct

▪ repetition of objects is not allowed

▪ order is NOT important

11
Permutations of Non-distinct Objects

▪ Number of permutations of 𝑛 objects of which 𝑛1 are of one kind, 𝑛2 are of a second kind,…, and 𝑛𝑘 are of a 𝑘-th
kind is given by:
𝑛!
𝑛1 ! 𝑛2 ! ⋯ 𝑛𝑘 !
▪ where: 𝑛 = 𝑛1 + 𝑛2 + ⋯ + 𝑛𝑘

▪ Example: How many different vertical arrangements are there of 10 flags if 5 are white, 3 are blue, and 2 are red?

▪ Solution: We seek the number of permutations of 10 objects, of which 5 are of one kind (white), 3 are of a second
kind (blue), and 2 are of a third kind (red)

▪ Using formula above, we find:

10! 10 × 9 × 8 × 7 × 6
= = 2,520
5! × 3! × 2! (3 × 2 × 1)(2 × 1)

12
Probability Axioms and Rules
Disjoint Events

▪ Notation: Venn Diagram

▪ 𝑃 𝐸 or 𝐹 = 𝑃 𝐸 ∪ 𝐹 = 𝑃(𝐸 + 𝐹)
▪ 𝑃 𝐸 and 𝐹 = 𝑃 𝐸 ∩ 𝐹 = 𝑃(𝐸 ⋅ 𝐹)
𝑆 𝐴
▪ Disjoint, a.k.a. mutually exclusive events, have
no outcomes in common
▪ Example: 𝐴 and 𝐷, 𝐶 and 𝐷 𝐷
𝐵 𝐶
▪ Probability of disjoint events:
𝐴 and 𝐷 disjoint ⇔ 𝐴 ∩ 𝐷 = ⇔𝑃 𝐴∩𝐷 =0

T. Landolsi - Stochastic Systems 14

Disjoint Events: Example

▪ Randomly select chip from bag containing ten chips

labeled: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
▪ 𝐸 event “choose number  2” 𝑆
{3,4,5,6,7}
▪ 𝐹 event “choose number  8”
▪ Are 𝐸 and 𝐹 disjoint i.e. mutually exclusive?
𝐸 = {0,1,2}
▪ 𝐸 and 𝐹 are disjoint (see figure) since there is no 𝐹 = {8,9}
intersection between them
▪ if 𝐸 occurs then 𝐹 cannot occur

15
Axioms of Probability

1. Probability of any event 𝐸 must be:
0 ≤ 𝑃(𝐸) ≤ 1
▪ impossible event  its probability is 0
▪ certain event  its probability is 1
3. Addition rule for disjoint events:
𝐸 ∩ 𝐹 = 0 ⇒ 𝑃 𝐸 ∪ 𝐹 = 𝑃 𝐸 + 𝑃(𝐹)
4. General addition rule:
𝑃 𝐸∪ 𝐹 =𝑃 𝐸 +𝑃 𝐹 −𝑃 𝐸∩𝐹

2. Sum of prob. of all outcomes = 1

▪ Note: 𝑛 ≥ 2 pairwise disjoint events:
for sample space 𝑆 = {𝑒1 , 𝑒2 , … , 𝑒𝑛 }:
𝑃 𝑒1 ∪ 𝑒2 ∪ ⋯ 𝑒𝑛 = 𝑃 𝑒1 + 𝑃 𝑒2 + ⋯ 𝑃(𝑒𝑛 )
𝑃 𝑒1 + 𝑃 𝑒2 + ⋯ + 𝑃 𝑒𝑛 = 1

T. Landolsi - Stochastic Systems 16

General Addition Rule: Example

▪ Throw pair of dice one after another

▪ 𝐸 = “the first die is a two”
▪ 𝐹 = “the sum of the dice is less than or equal to 5”
▪ Find 𝑃(𝐸 ∪ 𝐹)

▪ Solution:
▪ 𝐸 = {(2,1); (2,2); (2,3); (2,4); (2,5); (2,6)}
▪ 𝐹 = {(1,1); (1,2); (1,3); (1,4); (2,1); (2,2); (2,3); (3,1); (3,2); (4,1)}
13
⇒ 𝑃 𝐸∪𝐹 = 𝑃 𝐸 +𝑃 𝐹 −𝑃 𝐸∩𝐹 =
36

T. Landolsi - Stochastic Systems 17

Complementary Events

ത complement of 𝐸
▪ 𝐸: Venn Diagram

▪ composed of all 𝑆 outcomes not part of 𝐸

▪ sometimes noted 𝐸′ or 𝐸 c
𝑆
▪ Complement probability rule: 𝐸
𝑃 𝐸ത = 1 − 𝑃 𝐸

▪ Rule proves very handy in many situations

ത outcomes in 𝑆 not part of 𝐸

𝐸:

T. Landolsi - Stochastic Systems 18

Complementary Events: Example-1

Problem Solution

▪ Two PCs exchange file over Internet
▪ 1. Suppose that prob. that file is erroneous when it
reaches destination PC is: 𝑃 𝐹
▪ what is probability that file is received error-free?
▪ 1. Event “error-free file” is complementary to event
“erroneous file”
𝑃 𝐹ത = 1 − 𝑃 𝐹
▪ 2. error-free file means all its 𝑀 bits are error-free
▪ Prob. that one bit is error-free is: 1 − 𝑃 𝐵

▪ 2. suppose that prob. of bit error is 𝑃 𝐵 ▪ Prob. that all 𝑀 bits are error-free is: 𝑃 𝐹ത = 1 − 𝑃 𝐵 𝑀

𝑀
▪ Prob. that file is erroneous is: 𝑃 𝐹 = 1 − 1 − 𝑃 𝐵
▪ what is prob. that file with 𝑀 bits is received error-free?

▪ what is prob. that file is erroneous 𝑃 𝐹 , in terms of 𝑃 𝐵 ?

T. Landolsi - Stochastic Systems 19

Complementary Events: Example-2
Time (min) Frequency
T<5 24,358
Problem
5≤T≤9 39,112
▪ Data represent travel time to work for residents of city A 10 ≤ T ≤ 14 62,124
15 ≤ T ≤ 19 72,854

▪ What is probability that randomly selected city resident 20 ≤ T ≤ 24 74,386

has travel time < 90 minutes? 25 ≤ T ≤ 29 30,099
30 ≤ T ≤ 34 45,043
▪ Note: Total no. of residents: 24,358 + ⋯ = 393,186
35 ≤ T ≤ 39 11,169
40 ≤ T ≤ 44 8,045
▪ Solution: Complement rule → 𝑃(𝑇 < 90) = 0.988
45 ≤ T ≤ 59 15,650
60 ≤ T ≤ 89 5,451
T ≥ 90 4,895

T. Landolsi - Stochastic Systems 20