4: Probability

Acronyms
b

n
N
p
pdf
pmf
RV

x
X

binomial
expected value [parameter]
number of trials [parameter]
normal
probability of success [parameter]
probability density function
probability mass function
random variable
standard deviation [parameter]
value for random variable X (e.g., observed number of successes for a binomial random variable)
random variable X

What is probability?
The probability of an event is its relative frequency out of all possibilities events or its expected relative
frequency in the long run. If an event occurs x times out of n, then its probability will converge on X n
as n becomes infinitely large. For example, if we flip a coin many times, we expect to see half the flips
turn up heads.
Note that when n is small, the observed relative frequency of an event will not be a reliable reflection of
its probability. However, as the number of observations n increases, the observed frequency becomes a
more reliable reflection of the probability. For example, if a coin is flipped 10 times, there is no guarantee
that it will turn up as exactly 5 heads every time. However, if the coin is flipped 10,000 times, chances are
pretty good that the proportions of heads will be pretty close to 0.5.

Random variables (RVs)

A random variable (RV) is a quantity take of various values depending on chance. There are two types
of random variables: (1) discrete random variables and (2) continuous random variables.

Discrete random variables form a countable set of outcomes. We will study binomial random
variables as a way to familiarize ourselves with discrete random variable.

Continuous random variables form an infinite continuum of possible outcomes. We will study
normal (Gaussian) random variables as a way to familiarize ourselves with continuous random
variables.

Page 4.1 (C:\USERS\B. BURT GERSTMAN\DROPBOX\STATPRIMER\PROBABILITY.DOCX Print date: 3/8/16)

Binomial random variables

Definition
Binomial random variables are discrete RVs of counts that describe the number of successes (X) in n
independent Bernoulli trials, a where each Bernoulli trial has the probability of success p.
Binomial random variables have two parameters:
n the number of independent Bernoulli trials
p the consistent probability of success for each trial
EXAMPLE. As an example of a binomial RV, consider the number of successful treatments (X) in 3
patients (n) where the probability of success in each instance p is 0.25. X can take on the discrete values
of 0, 1, 2, or 3.

Notation
Let b represent the binomial distribution and ~ represent distributed as. Thus, X~b(n, p) is said
random variable X is distributed as a binomial random variable with parameters n and p. The binomial
random variable in our example is X~ b(3, .25).
Let Pr(X = x) represent the probability random variable X takes on the value x. For example, Pr(X = 0)
the probability of 0 successes for random variable X.
Let Pr(X x) represent the probability random variable X takes on a value x or less. This is the
cumulative probability of the event.

Probability mass function (pmf)

DEFINITION. The probability mass function (pmf) of a discrete random variable assigns probabilities
for all possible outcomes.
EXAMPLE. The pmf for X~b(3, .25) is shown in Table 1. The exact probabilities of events are shown in
the second column. The cumulative probability is shown in the third column.
TABLE 1. The pmf for X~b(3, .25).
X
Number of successes
0 (event A)
1 (event B)
2 (event C)
3 (event D)

Pr(X = x)
Probability
0.4219
0.4219
0.1406
0.0156

Pr(X x)
Cumulative Probability
0.4219
0.8438
0.9844
1.0000

How we calculated these probabilities is not currently the issue. Instead, let us focus on meaning. The
above pmf states that for X~b(3, .25) we expect to see 0 successes 0.4219 of the time, 1 success 0.4219 of
the time, 2 successes 0.1406 of the time, and 3 successes 0.0156 of the time.
a

A Bernoulli trial is a random event that can take on one of two possible outcomes: success or failure.

Page 4.2 (C:\USERS\B. BURT GERSTMAN\DROPBOX\STATPRIMER\PROBABILITY.DOCX Print date: 3/8/16)

COMPUTATION
For now we provide pmfs for you.
If you need to calculate binomial probabilities, use this app http://www.di-mgt.com.au/binomialcalculator.html.
If you are curious or for some reason you need to calculate binomial probabilities by hand, use
the formulas in Chapter 6 of my textbook Basic Biostatistics for Public Health Practice.

Elementary rules for working with probabilities

Let:

A event A (e.g., let A represent 0 success out of 3 in our Example).

B event B (e.g., let B represent 1 success out of 3).
Pr(A) the probability of event A. (e.g., Pr(A) = 0.4129).
the complement of event A not A.

the union of events (e.g., A B A or B).

Rule 1: Probabilities can be no less than 0% and no more than 100%. an event with probability 0 never
occurs and an event with probability 1 always occurs
0 Pr(A) 1
Note that an all the events in Table 1 obey this rule.
Rule 2: All possible outcomes taken together have probability exactly equal to 1.
Pr(all possible outcomes) = 1
Note that in Table 1, Pr(all possible outcomes) = 0.4129 + 0.4129 + .1406 + 0.0156 = 1.
Rule 3: When two events are disjoint (cannot occur together), the probability of their union is the sum of
their individual probabilities.
Pr(A B) = Pr(A) + Pr(B), if A and B are disjoint
In Table 1 let A 0 successes and A 1 success. Pr(A B) = 0.4219 + 0.4219 = 0.8438.
Rule 4: The probability of a complement is equal to 1 minus the probability of the event.
Pr() = 1 Pr(A)
In Table 1, (1, 2, or 3 successes) and Pr() = 1 0.4219 = 0.5781.

Page 4.3 (C:\USERS\B. BURT GERSTMAN\DROPBOX\STATPRIMER\PROBABILITY.DOCX Print date: 3/8/16)

The area under the curve (AUC) concept

Probability mass functions (pmfs) can be drawn. For our Example (pmf in Table 1):

X~BINOMIAL(3,.25)
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0

0.4219

0.4219
0.1406

X=0

X=1

X=2

0.0156
X=3

Figure 1. X~b(3, .25).

On the horizontal axis, the first bar stretches from 0 to 1. Therefore, this rectangle has base = 1. It has
height = 0.4219. Thus, the area of this bar = h b = 0.4219 1 = 0.4219 = Pr(X = 0).
The second bar also has a base of 1 (from 1 to 2), height of 0.4219, and area = h b = 0.4219 1 =
0.4219. This corresponds to Pr(X = 1).
The combined area of the first two bars = Pr(X = 0) or Pr(X = 1) = 0.4219 + 0.4219 = 0.8438.
In fact the area between any two values is equal to the probability of obtaining an outcome between these
two values. This fact is referred to as the area under the curve (AUC) rule.
We can also use the rule of complements to find AUCs (probabilities).
Let A 0 successes.
Therefore one or more successes.
By the rule of complements, Pr() = 1 0.4219 = 0.5781.
If you add up the Pr(X = 1) + Pr(X = 2) + Pr(X = 3) you will see that this AUC is also equal to
0.5791.

Page 4.4 (C:\USERS\B. BURT GERSTMAN\DROPBOX\STATPRIMER\PROBABILITY.DOCX Print date: 3/8/16)

Normal (Gaussian) random variables

Probability density functions (pdf)
We will use normal random variables as a venue to introduce continuous random variables. While
discrete random variables are characterized with chunky pmfs, continuous random variables are
represented with smooth probability density functions (pdf). The pdf cannot be shown in a table but it
can be drawn.
Normal pdfs will be recognized by their symmetrical bell shape. In addition, they are characterized by
two parameters, (mu) and (sigma).
the pdfs mean or expected value
the pdfs standard deviation (mean root deviation)
Note: there is no one single normal pdf! When changes, the location of the pdf changes

Figure 2. 1 < 2
identifies the pdfs spread around the mean. When changes, the pdf changes:

Figure 3. 1 > 2

Page 4.5 (C:\USERS\B. BURT GERSTMAN\DROPBOX\STATPRIMER\PROBABILITY.DOCX Print date: 3/8/16)

 and are the analogues of and s for data distributions. However, they do not describe a data
distribution. They describe the pdf, which is distinct from data.

You can get an idea of the size of on a normal pdf by identifying the curves points of inflection. This
is where the curve begins to change slope. Trace the normal curve with your finger. You are skiing
either the left slope or right slope. The point at which the slope begins to flatten is a point of inflection.

Figure 4. Points of inflection.

Note that the left point of inflection marks the location of and the right point of inflection marks the
location of .

Normal probabilities, AUC, and the 68-95-99.7 rule

Probabilities for continuous random variables cannot easily be shown in a table (for a couple of reasons).
They can be envisions in a drawing however, where they are represented as the area under the curve
(AUC)see prior section in the chapter about AUCs!
The 68-95-99.7 rule will help you visualized AUCs from normal curves. This rule says:

68% of the AUC for normal RVs lies in the region

95% of the AUC for normal RVs lies in the region 2
99.7% of the AUC for normal RVs lies in the region 3

This applies to normal pdfs only!

Page 4.6 (C:\USERS\B. BURT GERSTMAN\DROPBOX\STATPRIMER\PROBABILITY.DOCX Print date: 3/8/16)

Visually, the 95 part of the rule looks like this:

Think in terms of these landmarks:

Although and vary from one normal random variable to the next, you can use these 68-95-99.7 rule to
identify normal landmarks and approximate AUCs as long as you keep in mind that (a) probabilities
are AUCs, (b) the total AUC under the normal curve = 1 (rule 2), and (c) the normal values lie on the
horizontal axis. You are college students (or top high school students), so it is assumed you can make this
mental link.
EXAMPLE. The Wechsler Intelligence Scale is used to measure intelligence. It is calibrated to produce a
Normal distribution with = 100 and = 15 within each age group.
NOTATION. Let X~N(, ) represent a normal random variable with mean and standard deviation .
Using this notation, Wechsler Intelligence scale scores is represented X~N(100, 15).
The 689599.7 rule states that for X~N(100, 15):

68% of the AUC lies in the range = 100 15 = 85 to 115

95% of the AUC lies in the range 2 = 100 (2)(15) = 70 to 130
99.7% of the AUC lies in the range 3 = 100 (3)(15) = 55 to 145

Page 4.7 (C:\USERS\B. BURT GERSTMAN\DROPBOX\STATPRIMER\PROBABILITY.DOCX Print date: 3/8/16)

It helps to label the x-axis to orient these AUCs (probabilities) and normal values. This next figure shows
the AUCs for X~N(100, 15) that captures a left tail AUC of 0.025, a right tail AUC of 0.025, and middle
AUC area of 0.95. Notice the center of the curve on and the landmarks 1, 2, 3 landmarks on the
horizontal axis. We can see that 0.025 of the AUC is to the left of 70 and 0.025 is to the right of 130. 0.95
of the AUC is between 70 and 130.

Finding normal probabilities associated with normal values

Keep in mind that we are focusing on meaning and not on mechanical calculations. The key concept here
is the AUC between any two points corresponds to the probability of that observation.
This applet will calculate probabilities associated with any two points on any X~N(, ):
http://onlinestatbook.com/2/calculators/normal_dist.html. Below I have plugged in values for X~(100,15)
to determine the AUC between 130 and (right tail). Note the corresponding AUC (probability) is
0.0228, which rounds to 2.5%.

Page 4.8 (C:\USERS\B. BURT GERSTMAN\DROPBOX\STATPRIMER\PROBABILITY.DOCX Print date: 3/8/16)