Probability

Lets start with some definitions

Suppose you throw a pair of dice and add the numbers facing up (experiment) When you repeat this experiment long enough you will find out that the set of all possible outcomes is S = {2,3,4,,12}. S is called the sample space. Sometimes you are interested in a subset of S. For example, you may want to know, in the example above, the set of all possible even outcomes. We usually refer to such subsets as events (E). In this case E = {2,4,6,8,10,12}. Question: A coin is tossed three times and number of Heads are noted. What is the sample space for this experiment? What is the set of all outcomes where heads come up at least twice? S = {0,1,2,3} E = {HHH, HHT, HTH, THH} and E = {TTT, THT, TTH, HTT} E U E = S and E E =

Another example
A pair of dice (one red, one green) is thrown and the number facing up on each die is noted, let E be the event that the sum of the numbers is 4, and let F be the event that the sum is an odd number. a) The event F' is: {(1, 1), (1, 3), (1, 5), (2, 2), (2, 4), (2, 6), (3, 1), (3, 3), (3, 5), (4, 2), (4, 4), (4, 6), (5, 1), (5, 3), (5, 5), (6, 2), (6, 4), (6, 6)} b) E F' is the event: {(1, 3), (2, 2), (3, 1)} c) E' U F is the event: The sum of the numbers is any number other than 4

Estimated probability
a) The frequency of the event E [denoted as fr(E)] is the number of times the event E occurs. Example: Toss a coin 20 times. If heads comes up 13 times, then the frequency of the event that heads comes up is fr(E) = 13. b) The relative frequency or estimated probability of the event E is the fraction of times E occurs: P(E) = fr(E) / N. Referring to the example above, the estimated probability of the event that heads comes up is 13/20=65%.

c) The number of times that the experiment is performed is called the sample size (N).
The experiment above was performed 20 times, so the sample size is N = 20.

Estimated probability distribution

d) The collection of the estimated probabilities of all the outcomes is the estimated probability distribution. Example: If 10 rolls of a die resulted in the outcomes 2, 1, 4, 4, 5, 6, 1, 2, 2, 1, the associated estimated probability distribution is the one shown in the following table.
Outcome Probability 1 0.3 2 0.3 3 0 4 0.2 5 0.1 6 0.1

Properties of probability distributions

Let S = {s1, s2, ... , sn} be a sample space and let P(si) be the estimated probability of the event {si}. Then: 0 P(si) 1 The estimated probability of each outcome is a number between 0 and 1.

P(s1) + P(s2) + .. + P(sn)=1 The estimated probabilities of all the outcomes add up to 1.

If E = {e1, e2, ..., er}, then P(E) = P(e1) + P(e2) + ... + P(er). The estimated probability of an event E is the sum of the estimated probabilities of the individual outcomes in E.

Theoretical probability
Definition: The theoretical probability, or probability, P(E), of an event E is the fraction of times we expect E to occur if we repeat the same experiment over and over. Relationship with estimated probability: The estimated probability approaches the theoretical probability as the number of trials gets larger and larger. Thus, estimated probability is an approximation, or estimate, of theoretical probability. Determining theoretical probability: Theoretical probability is determined analytically, that is, by using our knowledge about the nature of the experiment rather than through actual experimentation. The best we can obtain through actual experimentation is an estimate of the theoretical probability (hence the term "estimated probability"). Example: Toss a fair coin and observe the uppermost side. Since we expect that heads is as likely to come up as tails, we conclude that the theoretical probability distribution is: P(H) = P(T) = 1/2. Similarly, if you roll a fair dice: P(1)=P(2)= . . . = P(6) = 1/6.

Probability distributions
Formal definition: A finite sample space is just any old finite set S = {s1, s2, ..., sn}. A probability distribution is an assignment of a number P(si) to each outcome si in a sample space S ={s1, s2, ... , sn} such that 0 P(si) 1 The estimated probability of each outcome is a number between 0 and 1. P(s1) + P(s2) + .. + P(sn)=1 The estimated probabilities of all the outcomes add up to 1. A few more facts to note: To get the probability of an event (recall that an event is a set of outcomes) all we do is add the probabilities of the individual outcomes. If P(E) = 0, we call E an impossible event.

An example
Fill in the missing probabilities, given that P(6) = 3P(1). What is the probability that an even number will come up?
Outcome Probability 1 ? 2 0.125 3 0.125 4 0.125 5 0.125 6 ?

Addition principle
a) If E and F are mutually exclusive events, then P(E U F) = P(E) + P(F). Example: Cast two dice and add the numbers facing up. E: the sum is 5; F: the sum is even. Since these are mutually exclusive events, the probability that the sum is either 5 or even is: P(E U F) = P(E) + P(F) = 1/9 + 1/2 = 11/18. b) If E and F are NOT mutually exclusive events, then we must use the more general formula: P(E U F) = P(E) + P(F) - P(E F). Example: Cast a single dice and note the number facing up. E: the outcome is 1, 2 or 3; so P(E) = 1/2; F: the outcome is even; P(E) = 1/2

E F = {2}; P(E F) = 1/6.

P(E U F)=P(E)+P(F)-P(E F) = 1/9 + 1/2 1/6 = 11/18.

Other principles
P(E') = 1 - P(E) P(S) = 1 P(O) = 1 P(S) = 0

Conditional probability
Quick illustration: Suppose you cast two dice; one red and one green. Then the probability of getting "bulls eyes" (two ones) is 1/36. However, if, after casting the dice, you ascertain that the green die shows a one (but know nothing about the red die), then there is a 1/6 change that both of them will be one. In other words, the probability of "bulls eyes" changes if you have partial information, and we refer to this (altered) probability as "conditional probability".

Example
Used cream Skin improved No improvement TOTAL 800 400 1,200 Placebo 600 200 800 TOTAL 1,400 600 2,000

The table above shows fictitious trial results of a new acne cream.
The sample size (total number of people in the study) is: 2,000 The estimated probability that someone's skin improved (regardless of which skin cream was used) is: 1,400/2,000 = 70%. (unconditional probability) The experimental probability that someone's skin improved, given that they used the new acne cream is: 800/1,200 = 66.7% (conditional probability)

Formally
The probability of the event E, given the event F, and written P(E|F) (probability of E, given F) If E and F are not mutually exclusive events, then the probability of E given F is

P( E | F )

P( E F ) P( F )

If all outcomes are equally likely, then we can also use the alternative formula

P( E | F )

fr ( E F ) fr ( F )

Another example
You have invested in Home-Clone Inc. stocks, as you suspect that the company's "Clonea-Sibling" kit will shortly be approved by the FDA. There is an 80% chance that FDA approval will be given, and a 95% chance that the value of the stock you hold will double if FDA approval is given. What is the probability that the value of the stock you hold will double? F: FDA approval, E: Price doubles P(E|F) = 95%, P(F) = 80%, P(E) = ? Using the formula above: 0.95 = 0.8 * P(E)

P(E) = 76%.

Independent Events
If E and F are independent if any one of the following equations holds: P(E|F) = P(E) P(F|E) = P(F) P(EF) = P(E) * P(F) Example: You throw two fair dice, one green and one red, and observe the numbers uppermost. A: the event that their sum is 7; P(E) = n(E)/36 = 6/36 = 1/6 B: the event that the red die shows an even number; P(F) = 1/2

P(AB) = P((1, 6), (3, 4), (5, 2)) = 3/36 = 1/12.

Since P(AB) = P(A) * P(B), A and B are independent.

Bayes Theorem
Bayes' theorem gives us a way of calculating P(E|F) from a knowledge of P(F|E).

P( F | E )

P( E | F ) * P( F ) P( E | F ) * P( F ) P( E | F ' ) * P( F ' )

Example: A manufacturer claims that its drug test will detect steroid use (that is, show positive for an athlete who uses steroids) 95% of the time. Your friend on the rugby team has just tested positive. What is the probability that he uses steroids is? E = the event that an anabolic steroid detection test gives a positive result F = the event that the athlete uses steroids P(E|F) = 95%, P(F|E) = ? To answer this, we need to know: a) P(F) (the probability that an athlete on the team is using steroids) b) P(E|F') (the probability of a false positive result).

Example continues
Suppose, 15% of all steroid-free individuals also test positive [P(E|F') =15%]. Also, 10% of the rugby team members use steroids [P(F)=10%]. Your friend on the rugby team has just tested positive. The probability that he uses steroids is:

P( F | E )

0.95 * 0.10 0.413 0.95 * 0.10 0.15 * 0.90

Another example
As accounts manager in your company, you classify 75% of your customers as "good credit" and the rest as "risky credit" depending on their credit rating. Customers in the "risky" category allow their accounts to go overdue 50% of the time on average, whereas those in the "good" category allow their accounts to become overdue only 10% of the time. What percentage of overdue accounts is held by customers in the "risky credit" category? F = the event that a customer is classified as risky credit E = the event that the customer account goes overdue P(F) = 25%, P(E/F) = 50%, P(E/F) = 10%, P(F/E)=? P(F|E) = [50% * 25%] / [50% * 25% + 10% * 75%] = 12.5% / 20% = 62.5%