Probablity

Pr obabi l i t y and St at i st i cs ( CENG 2007) Addi s Ababa Technol ogy I nst i t ut e
Compi led by: Di baba B./ di boobayu@gmai l.com/ ~ 45 ~

CHAPTER FOUR
4. BASIC PROBABILITY CONCEPT
4.1. Deterministic and non-deterministic models
4.2. Review of set theory: sets, union, intersection, complementation, De-Morgans rules
4.3. Random experiments, sample space and events
4.4. Counting techniques
4.5. Definitions of probability
4.6. Derived theorems of probability
4.7. Conditional probability and independency
4.8. Multiplication theorem, Bayes Theorem and Total probability theorem
4.9. Independent Events

4.1. Deterministic and non-deterministic models
Models are integral parts of both social and natural sciences. In both cases we construct or fit models to
represent the interrelationship between two or more variables. Particularly, in the fields like Statistics and
Economics models are fitted for the sake of forecasting. I It t i is s p po os ss si ib bl le e t to o c cl la as ss si if fy y m mo od de el ls s i in n t to o d di if ff fe er re en nt t
g gr ro ou up ps s b ba as se ed d o on n v va ar ri ie ed d a at tt tr ri ib bu ut te es s o or r c cr ri it te er ri ia a. . B Ba as se ed d o on n t th he e t ty yp pe e o of f e ex xp pe er ri im me en nt t f fo or r w wh hi ic ch h w we e f fi it t t th he e m mo od de el l, , w we e
c cl la as ss si if fy y m mo od de el l a as s D De et te er rm mi in ni is st ti ic c a an nd d N No on n- -d de et te er rm mi in ni is st ti ic c m mo od de el ls s. .
Definition: Experiment is any activity (process or action) that we intended to do under certain condition to
obtain a well defined results, usually called the outcome of an experiment. The possible results of an
experiment may be one or more. Based on the number of possible results, we may classify an experiment as
Deterministic and Non-deterministic experiment.
A- Deterministic experiment: the experiment for which the outcomes can be predicted in advance and is
known prior to its conduct is called a deterministic experiment. For this type of experiment we have only
one possible result (certain and unique). The result of an experiment is assumed to be dependent on the
condition under which an experiment is performed. A mathematical form of equations to be defined on
this experiment is called a deterministic model.
Definition: Deterministic model is a model that stipulates the condition under which an experiment is
performed determine the outcome of that experiment.
Examples: An experiment conducted to verify the Newton Laws of Motion: F = ma; an experiment
conducted to determine the economic law of demand:
t
= o + bP
t
where Q is a quantity demand, P is
the price and t is a time; etc
Note: Most of the experiments that we conducted to verify science law is an example of deterministic
experiment.
B- Non-deterministic experiment: this is an experiment for which the outcome of a given trial cannot
predict in advance prior to its conduct. We also call this experiment an unpredictable or probabilistic or
stochastic or Random Experiment. Usually the result of this experiment is subjected to chance and is
possible more than one. In this experiment, whatever the condition under which an experiment is
performed one cannot tell with certainty which outcome occur at any particular execution of an
experiment but we can list those outcomes.
This types of experiments are characterized by the following three properties:
I. The experiment is repeatable under identical conditions.
II. The outcome in any particular trial is variable; dependent on some chance or random mechanism.
III. If the experiment is repeated a large number of times, then some regularity becomes apparent in the
outcome obtained and the mathematical form of equations to be defined on those experiment to
handle those outcome regularity is called a Non-deterministic model.

Definition: Non-deterministic model is a model in which a chance plays an important role on the outcome
of an experiment.
Example: The experiment of tossing a coin: there are two possible outcomes i.e. getting a Head or Tail.
Though we know the possible outcomes, we cannot for sure predict that we can get a tail if we throw the
coin in a particular manner and so on.
Note: the quantitative measure of a certainty regarding one or more outcomes of a random experiment is
called a Probability.
Exercise: I Id de en nt ti if fy y t th he e f fo ol ll lo ow wi in ng g m mo od de el ls s a as s d de et te er rm mi in ni is st ti ic c o or r N No on n- -d de et te er rm mi in ni is st ti ic c
a a) ) E Eq qu ua at ti io on n o of f a a s st tr ra ai ig gh ht t l li in ne e. .
b b) ) V Va ar ri ia ab bl le e c co os st t= =( (C Co os st t p pe er r u un ni it t) )( (Q Qu ua an nt ti it ty y p pr ro od du uc ce ed d) )
c c) ) Y Y
i i= =A AX X
i i+ +e e
i i w wh he er re e e e
i i i is s t th he e e er rr ro or r t te er rm m ( (o or r r ra an nd do om m c co om mp po on ne en nt t) ) a as ss so oc ci ia at te ed d w wi it th h t th he e i i
t th h

o ob bs se er rv va at ti io on n. .
4.2. Review of set theory: sets, union, intersection, complementation, De-Morgans rules
D De ef fi in ni it ti io on n a an nd d T Ty yp pe es s o of f S Se et ts s
S Se et t: : A A s se et t i is s a a w we el ll l- -d de ef fi in ne ed d c co ol ll le ec ct ti io on n o or r l li is st t o of f o ob bj je ec ct ts s. .
A An n o ob bj je ec ct t t th ha at t b be el lo on ng gs s t to o a a p pa ar rt ti ic cu ul la ar r s se et t i is s c ca al ll le ed d a an n e el le em me en nt t. . S Se et ts s a ar re e u us su ua al ll ly y d de en no ot te ed d b by y c ca ap pi it ta al l l le et tt te er rs s ( (A A, ,
B B, , C C e et tc c) ). . O On n t th he e o ot th he er r h ha an nd d, , e el le em me en nt ts s o of f s se et ts s a ar re e u us su ua al ll ly y d de en no ot te ed d b by y s sm ma al ll l l le et tt te er rs s ( (a a, , b b, , c c e et tc c) ). .
E Ex xa am mp pl le es s: : T Th he e s se et t o of f s st tu ud de en nt ts s i in n a a c cl la as ss s; ; t th he e s se et t o of f e ev ve en n n nu um mb be er rs s; ; t th he e s se et t o of f p po os ss si ib bl le e o ou ut tc co om me es s o of f a an n
e ex xp pe er ri im me en nt t; ; e et tc c. .
N No ot te e: : I If f X X b be el lo on ng gs s t to o s se et t A A w we e w wr ri it te e X X e e A A, , a an nd d i if f X X d do oe es s n no ot t b be el lo on ng g t to o s se et t A A w we e w wr ri it te e X X e e A A
T Tw wo o S Sp pe ec ci ia al l S Se et ts s
U Un ni iv ve er rs sa al l s se et t ( (U U) ): : U Un ni iv ve er rs sa al l s se et t i is s t th he e c co ol ll le ec ct ti io on n o of f a al ll l o ob bj je ec ct ts s u un nd de er r c co on ns si id de er ra at ti io on n. . U Un ni iv ve er rs sa al l s se et t f fo or r a a g gi iv ve en n
d di is sc cu us ss si io on n i is s f fi ix xe ed d a an nd d p pr re e- -d de et te er rm mi in ne ed d. .
E Ex xa am mp pl le e: : * * T Th he e s se et t o of f r re ea al l n nu um mb be er rs s c ca an n b be e s se ee en n t to o b be e u un ni iv ve er rs sa al l s se et t o of f n nu um mb be er rs s. .
* * C Co on ns si id de er r a an n e ex xp pe er ri im me en nt t o of f r ro ol ll li in ng g a a b ba al la an nc ce ed d d di ie e. . T Th he en n, , t th he e s se et t o of f a al ll l p po os ss si ib bl le e o ou ut tc co om me es s o of f t th hi is s
e ex xp pe er ri im me en nt t c ca an n b be e c co on ns si id de er re ed d a as s t th he e u un ni iv ve er rs sa al l s se et t. . U U= ={ {1 1, ,2 2, ,3 3, ,4 4, ,5 5, ,6 6} }
E Em mp pt ty y s se et t ( ( o or r ) ): : A A s se et t w wi it th h n no o e el le em me en nt t i is s c ca al ll le ed d e em mp pt ty y s se et t. .
E Ex xa am mp pl le e: : I In n t th he e e ex xp pe er ri im me en nt t o of f t to os ss si in ng g t tw wo o d di ic ce e a at t t th he e s sa am me e t ti im me e, , d de ef fi in ne e A A t to o b be e a a s se et t w wh ho os se e e el le em me en nt ts s a ar re e
p pa ai ir rs s o of f o ou ut tc co om me es s f fr ro om m t th he e t tw wo o d di ic ce e w wi it th h s su um m g gr re ea at te er r t th ha an n 2 20 0. . O Ob bv vi io ou us sl ly y, , t th he e s se et t A A i is s e em mp pt ty y s se et t b be ec ca au us se e t th he e
m ma ax xi im mu um m p po os ss si ib bl le e s su um m i is s 1 12 2, , w wh hi ic ch h i is s a at tt ta ai in ne ed d w wh he en n b bo ot th h t th he e o ou ut tc co om me es s a ar re e 6 6. .
S Su ub bs se et t: : A A _ _ B B i if ff f X X e e A A X X e e B B f fo or r a al ll l X X e el le em me en nt t i in n t th he e u un ni iv ve er rs sa al l s se et t. .
4 4. .2 2. .1 1 S Se et t o op pe er ra at ti io on ns s
O On ne e c ca an n i id de en nt ti if fy y t th hr re ee e b ba as si ic c s se et t o op pe er ra at ti io on ns s. . T Th he es se e a ar re e: :
a a . . C Co om mp pl le em me en nt t: : F Fo or r a an ny y s se et t A A, , t th he e c co om mp pl le em me en nt t o of f A A d de en no ot te ed d b by y A A
/ /
, , o or r A A
c c
o or r i is s g gi iv ve en n b by y

X X U U/ / X X A A
N No ot te e: : U U
/ /
= = | | ; ; | |
/ /
= = U U ; ; ( ( A A
/ /
) )
/ /
= = A A
b b. . U Un ni io on n: : G Gi iv ve en n t tw wo o s se et ts s A A a an nd d B B, , t th he e u un ni io on n o of f A A a an nd d B B d de en no ot te ed d b by y A A B B i is s t th he e s se et t o of f a al ll l e el le em me en nt ts s, , w wh hi ic ch h
b be el lo on ng g t to o s se et t A A o or r B B, , o or r b bo ot th h. .
A A B B = = X X: : X X A A X X B B ; ; h he er re e t th he e l lo og gi ic ca al l c co on nn ne ec ct ti iv ve e v v ( (O OR R) ) i is s u us se ed d i in n i it ts s i in nc cl lu us si iv ve e s se en ns se e. .
I In n g ge en ne er ra al l, , i if f w we e h ha av ve e a a s se eq qu ue en nc ce e o of f s se et ts s A A
1 1, ,
A A
2 2, , , , A A
n n t th he en n
A A
1 1
A A
2 2 A A
n n = = n} and 1 between i one least at for : {
1
i
n
i
i
A X U X A e e =
=

R Re ec ca al ll l t th ha at t A A | | = = A A a an nd d A A U U = = U U
c c. . I In nt te er rs se ec ct ti io on n: : F Fo or r a an ny y t tw wo o s se et ts s A A a an nd d B B t th he e i in nt te er rs se ec ct ti io on n o of f A A a an nd d B B i is s d de ef fi in ne ed d t to o b be e t th he e s se et t o of f a al ll l
e el le em me en nt ts s t th ha at t o oc cc cu ur r i in n b bo ot th h s se et t A A a an nd d a al ls so o s se et t B B. . S Sy ym mb bo ol li ic ca al ll ly y, , w we e w wr ri it te e
A A B B = = X X U U / / X X A A X X B B
A A
1 1

A A
2 2
A A
n n
= = n} and 1 between i all for : {
1
i
n
i
i
A X U X A e e =
=

S Se et ts s w wi it th h n no o i in nt te er rs se ec ct ti io on n a ar re e c ca al ll le ed d d di is sj jo oi in nt t s se et ts s. . A A a an nd d B B a ar re e d di is sj jo oi in nt t i if f A A B B = = | |. .
4 4. .2 2. .2 2 P Pr ro op pe er rt ti ie es s o of f s se et t o op pe er ra at ti io on ns s
I I. . C Co om mm mu ut ta at ti iv ve e l la aw w
i i) ) A A B B = = B B A A, , a an nd d
i ii i) ) A A B B = = B B A A
I II I. . A As ss so oc ci ia at ti iv ve e l la aw w
i i) ) A A ( (B B C C) ) = = ( (A A B B) ) C C, , a an nd d
i ii i) ) A A ( (B B C C) ) = = ( (A A B B) ) C C
I II II I. . D Di is st tr ri ib bu ut ti iv ve e l la aw w
i i) ) A A ( (B B C C) ) = = ( (A A B B) ) ( (A A C C) ), ,
i ii i) ) A A ( (B B C C) ) = = ( (A A B B) ) ( (A A C C) ), ,
M Mo or re e g ge en ne er ra al ll ly y, ,
( )
( )

n
i
i
n
i
i
n
i
i
n
i
i
B A B A
and B A B A
1 1
1 1
,
= =
= =
=
|
|
.
|
\
|
=
|
|
.
|
\
|

Compi led by: Di baba B./ di boobayu@gmai l.com/ ~ 48
~

I IV V. . D De em mo or rg ga an n s s l la aw w
( (A A B B) )
/ /
= = A A
/ /
B B
/ /
a an nd d ( (A A B B) )
/ /
= = A A
/ /
B B
/ /

M Mo or re e g ge en ne er ra al ll ly y, ,

n
i
i
n
i
i
n
i
i
n
i
i
A A and A A
1
'
'
1 1
'
'
1
,
= = = =
=
|
|
.
|
\
|
=
|
|
.
|
\
|

4.3. Random experiments, sample space and events
I In nt tr ro od du uc ct ti io on n
P Pe er rh ha ap ps s t th he e c co on nc ce ep pt t o of f s sa am mp pl le e s sp pa ac ce e a an nd d e ev ve en nt ts s m ma ay y b be e c co on ns si id de er re ed d a as s t th he e c co or rn ne er rs st to on ne es s f fo or r t th he e w wh ho ol le e
d di is sc cu us ss si io on n i in n p pr ro ob ba ab bi il li it ty y. . T Th he e c co on nc ce ep pt t o of f p pr ro ob ba ab bi il li it ty y c ca an n n ne ev ve er r b be e a ad dd dr re es ss se ed d w wi it th ho ou ut t f fu ur rn ni is sh hi in ng g t th he e i id de ea a o of f
s sa am mp pl le e s sp pa ac ce e a an nd d e ev ve en nt ts s. . T Th he e g go oo od d n ne ew ws s i is s t th ha at t o on ne e c ca an n e ea as si il ly y g gr ra as sp p t th he e c co on nc ce ep pt t o of f s sa am mp pl le e s sp pa ac ce e a an nd d e ev ve en nt ts s
p pr ro ov vi id de ed d t th he er re e i is s a a c cl le ea ar r u un nd de er rs st ta an nd di in ng g o of f u un ni iv ve er rs sa al l s se et t a an nd d s su ub bs se et ts s, , w wh hi ic ch h a ar re e e el le em me en nt ta ar ry y c co on nc ce ep pt ts s i in n s se et t
t th he eo or ry y. . W We e h ha av ve e a a p pe er rf fe ec ct t a an na al lo og gy y. . T Th he e c co on nc ce ep pt ts s o of f u un ni iv ve er rs sa al l s se et t a an nd d s su ub bs se et ts s i in n t th he e t th he eo or ry y o of f s se et ts s d di ir re ec ct tl ly y
c co or rr re es sp po on nd d, , r re es sp pe ec ct ti iv ve el ly y, , t to o t th he e c co on nc ce ep pt ts s s sa am mp pl le e s sp pa ac ce e a an nd d e ev ve en nt ts s w wh he en n i it t c co om me es s t to o t th he e t th he eo or ry y o of f
p pr ro ob ba ab bi il li it ty y. .
1 1. . R Ra an nd do om m E Ex xp pe er ri im me en nt t: : I It t i is s a an n e ex xp pe er ri im me en nt t t th ha at t c ca an n b be e r re ep pe ea at te ed d a an ny y n nu um mb be er r o of f t ti im me es s u un nd de er r s si im mi il la ar r
c co on nd di it ti io on ns s a an nd d i it t i is s p po os ss si ib bl le e t to o e en nu um me er ra at te e t th he e t to ot ta al l n nu um mb be er r o of f o ou ut tc co om me es s w wi it th h o ou ut t p pr re ed di ic ct ti in ng g a an n i in nd di iv vi id du ua al l
o ou ut t c co om me e. . R Ra an nd do om m e ex xp pe er ri im me en nt ts s a ar re e a as ss so oc ci ia at te ed d w wi it th h p pr ro ob ba ab bi il li it ty y m mo od de el ls s. . A As s t th he e e ex xp pe er ri im me en nt t i is s r re ep pe ea at te ed d a a
l la ar rg ge e n nu um mb be er r o of f t ti im me e a a c ce er rt ta ai in n p pa at tt te er rn ns s a ap pp pe ea ar r. .
S So om me e f fe ea at tu ur re es s o of f a a r ra an nd do om m e ex xp pe er ri im me en nt t
E Ea ac ch h e ex xp pe er ri im me en nt t i is s c ca ap pa ab bl le e o of f b be ei in ng g r re ep pe ea at te ed d i in nd de ef fi in ni it te el ly y u un nd de er r e es ss se en nt ti ia al ll ly y u un nc ch ha an ng ge ed d c co on nd di it ti io on ns s, ,
A Al lt th ho ou ug gh h w we e a ar re e, , i in n g ge en ne er ra al l, , u un na ab bl le e t to o d de et te er rm mi in ne e w wh hi ic ch h p po os ss si ib bl le e o ou ut tc co om me e w wi il ll l r re es su ul lt t i in n a a s si in ng gl le e t tr ri ia al l
o of f t th he e e ex xp pe er ri im me en nt t, , w we e a ar re e a ab bl le e t to o d de es sc cr ri ib be e t th he e s se et t o of f a al ll l p po os ss si ib bl le e o ou ut tc co om me es s. .
A As s t th he e e ex xp pe er ri im me en nt t i is s r re ep pe ea at te ed d a a l la ar rg ge e n nu um mb be er r o of f t ti im me es s a a d de ef fi in ni it te e p pa at tt te er rn n o or r r re eg gu ul la ar ri it ty y i is s e ex xp pe ec ct te ed d t to o
a ap pp pe ea ar r. . F Fr ro om m t th hi is s r re eg gu ul la ar ri it ty y w we e c ca an n c co on ns st tr ru uc ct t a a p pr re ec ci is se e m ma at th he em ma at ti ic ca al l m mo od de el l. .
S Sa am mp pl le e s sp pa ac ce e: : T Th he e s se et t o of f a al ll l p po os ss si ib bl le e o ou ut tc co om me es s o of f a a r ra an nd do om m e ex xp pe er ri im me en nt t. .
E Ex xa am mp pl le e: : C Co on ns si id de er r t th he e r ra an nd do om m e ex xp pe er ri im me en nt ts s g gi iv ve en n a ab bo ov ve e
i i) ) F Fo or r t th he e e ex xp pe er ri im me en nt t o of f f fl li ip pp pi in ng g a a c co oi in n t th he e p po os ss si ib bl le e o ou ut tc co om me es s a ar re e e ei it th he er r H He ea ad d ( (H H) ) o or r T Ta ai il l ( (T T) ). .
H He en nc ce e, , t th he e s sa am mp pl le e s sp pa ac ce e w wi il ll l b be e S S = ={ {H H, , T T } }
i ii i) ) F Fo or r t th he e e ex xp pe er ri im me en nt t o of f r ro ol ll li in ng g a a d di ie e t th he e p po os ss si ib bl le e o ou ut tc co om me es s a ar re e t th he e s si ix x s si id de es s o of f t th he e d di ie e t th ha at t a ar re e
n nu um mb be er re ed d f fr ro om m 1 1 t to o 6 6. . T Th hu us s, , t th he e s sa am mp pl le e s sp pa ac ce e w wi il ll l b be e S S= ={ {1 1, ,2 2, ,3 3, ,4 4, ,5 5, ,6 6} }
i ii ii i) ) T To os ss s a a c co oi in n t tw wi ic ce e a an nd d o ob bs se er rv ve e t th he e f fa ac ce e u up p. . T Th he e s sa am mp pl le e s sp pa ac ce e o of f t th hi is s e ex xp pe er ri im me en nt t i is s S S = ={ {H HH H, , H HT T, ,
T TH H, , T TT T } }. . W Wh ha at t w wo ou ul ld d b be e t th he e s sa am mp pl le e s sp pa ac ce e i if f y yo ou u w we er re e t to o t to os ss s t th hr re ee e t ti im me es s? ?
i iv v) ) F Fo or r t th he e e ex xp pe er ri im me en nt t o of f d dr ra aw wi in ng g a a c ca ar rd d f fr ro om m a a d de ec ck k o of f c ca ar rd ds s t th he e f fo ol ll lo ow wi in ng g s se et ts s c ca an n b be e p po os ss si ib bl le e
s sa am mp pl le e s sp pa ac ce es s. .
S S= ={ {H He ea ar rt t, , S Sp pi id de er r, , D Di ia am mo on nd d, , F Fl lo ow we er r} }
S S= ={ {1 1s s, , 2 2s s, , 3 3s s, , , , Q Qs s, , K Ks s, , j jo ok ke er rs s} }

S S= ={ {B Bl la ac ck k c ca ar rd d, , R Re ed d c ca ar rd d} }
A As s t th he e l la as st t c ca as se e o of f t th he e a ab bo ov ve e e ex xa am mp pl le e i in nd di ic ca at te es s i it t i is s p po os ss si ib bl le e t to o h ha av ve e m mo or re e t th ha an n o on ne e s sa am mp pl le e s sp pa ac ce e f fo or r a a g gi iv ve en n
r ra an nd do om m e ex xp pe er ri im me en nt t. . H He en nc ce e, , w we e c ca an n s st ta at te e t th he e f fo ol ll lo ow wi in ng g r re em ma ar rk k. .
S Sa am mp pl le e S Sp pa ac ce e ( (S S) ): : I Is s t th he e s se et t o of f a al ll l p po os ss si ib bl le e o ou ut tc co om me e o of f a a r ra an nd do om m e ex xp pe er ri im me en nt t. .
There are different types of sample space:-
Finite sample space: if they have a finite number of elements. Example:
- Tossing a coin. S={Heads, Tails}
- Throwing a die. S={1, 2, 3, 4, 5, 6}
- Throwing a coin and a die. S={H1, H2, H3, H4, H5, H6, T1, T2, T3, T4, T5, T6}.
Countably infinite sample space: if there is a one-to-one mapping between the elements of S and
the natural numbers;
- S={0, 1, 2, 3, }
- S={ -3, -2, -1, 0, 1, 2, 3, }
- S={p: p is prime} = {2, 3, 5, 7, 11, 13, 17, }
- S={1/r: r is a positive natural number} = {1/2, 1/3, 1/4, 1/5, }
Uncountable sample space: if there are an infinite number of elements in S and there is not a one-
to-one mapping between the elements and the natural numbers.
- S=(0, )
- S=(, )
- S={(x, y): 0 x 1; y>0}
2. Outcome: The result of a single trial of experiment
3. Event: It is a subset of sample space. It is a statement about one or more outcomes of a random
experiment. It is denoted by capital letter.
E Ex xa am mp pl le es s: :
I In n t th he e e ex xp pe er ri im me en nt t o of f f fl li ip pp pi in ng g a a d di ie e w we e c ca an n d de ef fi in ne e a an n e ev ve en nt t E E
1 1= =H He ea ad ds s s sh ho ow ws s u up p= ={ {H H} }
I In n t th he e e ex xp pe er ri im me en nt t o of f r ro ol ll li in ng g a a d di ie e w we e c ca an n d de ef fi in ne e a an n e ev ve en nt t C C
2 2= =T Th he e n nu um mb be er r i is s o od dd d { {1 1, ,3 3, ,5 5} }
I In n t to os ss si in ng g a a c co oi in n t tw wi ic ce e w we e c ca an n d de ef fi in ne e a an n e ev ve en nt t D D
1 1= =A At t l le ea as st t o on ne e h he ea ad d o oc cc cu ur rs s= ={ {H HH H, , T TT T, , T TH H} }
N No ot te e: :
If S has n members, then there are exactly 2
n
subsets or events.
Recall that any set is the subset of itself. Accordingly, the sample space S by itself is an event called
a sure event or certain event.
An event occurs if any one of its elements turns out to be the outcome of the experiment.
Empty set (|) is called an impossible event.
4 4. . M Mu ut tu ua al ll ly y E Ex xc cl lu us si iv ve e e ev ve en nt ts s: : T Tw wo o e ev ve en nt ts s A A a an nd d B B a ar re e s sa ai id d t to o b be e m mu ut tu ua al ll ly y e ex xc cl lu us si iv ve e i if f t th he ey y c ca an nn no ot t o oc cc cu ur r
s si im mu ul lt ta an ne eo ou us sl ly y ( (t to og ge et th he er r) ) o or r A A B B = = | |
E Ex xa am mp pl le e: : E Ex xp pe er ri im me en nt t - -T To os ss s a a c co oi in n t tw wi ic ce e
S S= = { {H HH H, , H HT T, , T TH H, , T TT T} }

L Le et t A A= = T Tw wo o h he ea ad ds s o oc cc cu ur r { {H HH H} }
B B= = T Tw wo o t ta ai il ls s o oc cc cu ur r { {T TT T} }
C C= = A At t l le ea as st t o on ne e h he ea ad d o oc cc cu ur r { {H HH H, , H HT T, , T TH H} }
A A B B= = | | A A a an nd d B B a ar re e m mu ut tu ua al ll ly y e ex xc cl lu us si iv ve e e ev ve en nt ts s
B B C C= = | | B B a an nd d C C a ar re e m mu ut tu ua al ll ly y e ex xc cl lu us si iv ve e e ev ve en nt ts s
A A C C= = { {H HH H} } A A a an nd d C C a ar re e n no ot t m mu ut tu ua al ll ly y e ex xc cl lu us si iv ve e
5. Independent Events: Two events are said to be independent if the occurrence of one does not affect
probability of the other occurring.
6. Dependent Events: Two events are dependent if the first event affects the outcome or occurrence of the
second event in a way the probability is changed.
7 7. . C Co om mb bi in na at ti io on n o of f e ev ve en nt ts s
a) Union: The event A B occurs if either A or B or both occur.
I If f A A
i i, , i i= =1 1, , 2 2, , 3 3 , , n n i is s a an ny y f fi in ni it te e c co ol ll le ec ct ti io on n o of f e ev ve en nt ts s, , t th he en n
n
i
i
A
1 =
o oc cc cu ur rs s i if f a at t l le ea as st t o on ne e o of f t th he e e ev ve en nt ts s
( (A A
i i) ) o oc cc cu ur rs s. .
b) Intersection: The event A B occurs if both A and B occur.
I If f A A
i i, , i i= =1 1, ,2 2, ,3 3, , , ,n n i is s a an ny y f fi in ni it te e c co ol ll le ec ct ti io on n o of f e ev ve en nt ts s, , t th he en n
n
i
i
A
1 =
o oc cc cu ur rs s i if f a al ll l A A
i i o oc cc cu ur r. .
c c) ) Complement of an Event: the complement of an event A means nonoccurrence of A and is denoted
by A
i
or A
c
or A
contains those points of the sample space which dont belong to A.

S
S
A A A A
/ /
= = S S
A A A A
/ /
= = |
4.4. Counting techniques
In order to calculate probabilities, we have to know
The number of elements of an event
The number of elements of the sample space.
That is in order to judge what is probable, we have to know what is possible.
In order to determine the number of outcomes, one can use several rules of counting.
The addition rule
The multiplication rule
Permutation rule
Combination rule
I. Addition Rule
Let E1 and E2 be mutually exclusive events (i.e. there are no common outcomes).
Let event E describe the situation where either event E1 or event E2 will occur.

A
A

The number of times event E will occur can be given by the expression:
n(E) = n(E1) + n(E2)
where
n(E) = Number of outcomes of event E
n(E1) = Number of outcomes of event E1
n(E2) = Number of outcomes of event E2
In general, if there are K procedures and the i
th
procedure can be performed in n
ways, i = 1, 2, . , k, then
the number of ways in which we may perform procedure 1 or 2 or.. or procedure k is given by n
1
+ n
2
+
. + n
k
= n
k
=1
, assuming no procedures may be performed together.
Example 1: Suppose that we are planning a trip and deciding between bus and train transportation. If there
are three bus routes and two train routes. How many different routes are available for the trip?
Solution: Let A= {an event for bus transportation}, B= {an event for train transportation}.
Then, we have n
1
= 3 and n
2
= 2; assuming that we are going to use only one route, by the addition rule
we have n
1
+ n
2
= 3 + 2 = 5 different possible routes are available for the trip.
Example 2
In how many ways can a number be chosen from 1 to 22 such that it is a multiple of 3 or 8?
Solution:
Here, E1 = multiples of 3:
E1 = {3, 6, 9,12, 15, 18, 21}
n(E1) = 7
E2 = multiples of 8:
E2 = {8, 16}
n(E2) = 2
Events E1 and E2 are mutually exclusive.
n(E) = n(E1) + n(E2) = 7 + 2 = 9
II. The Multiplication Rule
If there are K procedures and the i
th
procedure can be performed in n
ways, i = 1, 2, . , k, then the

procedure consisting of procedure 1 followed by procedure 2,, followed by procedure k may be
performed in n
1
n
2
n
k
= n
k
=1
ways.
Example 1: Suppose that there are three different types of meal for lunch and four different types of soft
drinks. How many choices of meal and soft drinks can be made?
Solution: n
1
= 3 onJ n
2
= 4 hence n
1
n
2
= 3 4 = 12 different choices of meal and soft drinks can be
made.
Example 2: How many two digit numerals can be written by choosing the tens digit from A={1,3,5,7,9}
and the units digit from B= {2,4,6,8}
Solution: The Selection Consists of two steps where the 1
st
can be made in 5 different ways for the tens
digits and for each of these the 2
nd
can be made in 4 different ways for the units digit, Hence the whole
selection one after the other can be made in 5 x 4 different ways. i.e. there are 20 two digit numerals
i.e. n
1
= 5 and n
2
= 4
Example 3: The digits 0, 1, 2, 3, and 4 are to be used in 4 digit identification card. How many different
cards are possible if
a) Repetitions are permitted.
b) Repetitions are not permitted.
Solution:
a) There are four steps

1
st
digit 2
nd
digit 3
rd
digit 4
th
digit
5 5 5 5
1. Selecting the 1
st
digit, this can be made in 5 ways.
2. Selecting the 2
nd
3. Selecting the 3
rd
4. Selecting the 4
th
Hence, n
1
n
2
n
3
n
4
= 5 5 5 5 = 625 different cards are possible.
b) There are four steps
1
st
digit 2
nd
digit 3
rd
digit 4
th
digit
5 4 3 2
1. Selecting the 1
st
2. Selecting the 2
nd
3. Selecting the 3
rd
4. Selecting the 4
th
Hence, n
1
n
2
n
3
n
4
= 5 4 3 2 = 120 different cards are possible.
Example 4: There are 6 roads between A and B and 4 roads between B and C.
a. In how many ways can one drive from A to C by way of B?
b. In how many ways can one drive from A to C and back to A, passing through B on both trip?
c. In how many ways one drive the circular trip described in (b) without using the same road more than
once?
Ans. 24, 576 and 360 resp
III. Permutation
An arrangement of n objects in a specified order is called permutation of the objects.
Permutation Rules:
1. The number of permutations of n distinct objects taken all together is n!
Where n! = n ( n 1) ( n 2) , , 2 1. Note that: 0! = 1
2. The arrangement of n objects in a specified order using r objects at a time is called the permutation of n
objects taken r objects at a time. It is written as n
P
r and the formula is
( )! r n
n!
= P
r n

3. The number of distinct permutation of n objects in which n
1
are alike, n
2
are alike ---- etc is
! * .... !* !*
2 1 n
n n n
n!

4. A set of n distinct objects can be arranged around a circle is (n-1)! ways.
Example 1: Suppose we have a letters A, B, C, and D
a) How many permutations are there taking all the four?
b) How many permutations are there two letters at a time?
Solution: a) Here n = 4, there are four distinct objects. Hence, there are 4! = 24 permutations.
b) Here n = 4 onJ r = 2. There are
( )
2
2
24
2 4
4
2 4
= =
!
!
= P permutation.
Example 2: How many different permutations can be made from the letters in the word
CORRECTION? Ans: 453600


Example 3: In how many ways can a party of 7 people arrange themselves?
a) In a row of 7 chairs?
b) Around a circular table?
c) How many ways are there if two of the persons are not allowed to sit next to each other? Ans.:
5040, 720 and 360 resp.
IV. Combination
A selection of objects with out regard to order is called combination.
Example: Given the letters A, B, C, and D list the permutation and combination for selecting two letters.
Solution:
Permutation Combination
AB BA CA DA
AC BC CB DB
AD BD CD DC
AB BC
AC BD
AD DC
Note that: in permutation AB is different from BA. But in combination AB is the same as BA.
Combination Rule
The number of combinations of r objects selected from n objects is denoted by
n
r
or (
n
) and is given by the

formula:
[
n
r
=
n!
( n r) ! r!

Example 1: In how many ways can a committee of 5 people be chosen out of 9 people?
Ans: 126 ways
Example 2: Among 15 clocks there are two defectives .In how many ways can an inspector chose three of
the clocks for inspection so that:
a) There is no restriction.
b) None of the defective clock is included.
c) Only one of the defective clocks is included.
d) Two of the defective clock is included.
Ans: 455, 286, 156, and 13 ways, respectively.
Example 3: A delegation of four students is selected each year from a college to attend the ESA meeting.
a. In haw many ways can the delegation be chosen if there are 12 eligible students?
b. In how many ways if two of the eligible students will not attend the meeting together?
c. In how many ways if two of the eligible students are married and will only attend the meeting
together?
495, 10C4=210 + 2C1*10C3=240=450
4.5. Definitions of probability
In any random experiment there is always uncertainty as to whether a particular event will or will not occur.
Thus, if A is an event associated with an experiment we cannot state with certainty that an event A will
occur or will not occur. Hence, it becomes very important to try to associate a number with the event A
which will measure how likely is the event A will occur in the defined sample space. This leads to the theory
of probability in which we are going to consider the quantitative measure of uncertainty.
Approaches to measuring probability
There are four different conceptual approaches to study probability theory. These are:

The classical approach.
The frequentist approach.
The subjective approach.
The axiomatic approach.
A. The classical approach
This approach is used when:
All outcomes are equally likely and mutually exclusive.
Total number of outcome is finite, say N.
Definition: If a random experiment with N equally likely outcomes is conducted and out of these N
A
outcomes are favorable to the event A, then the probability that event A occur denoted P (A) is defined as:
P( A) =
N
A
N
=
N0. out comes ar e favor abl e t o t he event A
Tot al nuber out comes
=
n( A)
n( S)

Example 1: A fair die is tossed once. What is the probability of getting
a) Number 4?
b) An odd number?
c) Number greater than 4?
d) Either 1 or 2 or . Or 6
Example 2: A box of 80 candles consists of 30 defective and 50 non defective candles. If 10 of these
candles are selected at random, what is the probability?
a) All will be defective.
b) 6 will be non defective
c) All will be non defective
Solution: Iotol sclcction = (
80
10
) = N = n( S)
Let A be the event that all will be defective.
The total way in which A will occur =(
30
10
)(
50
0
) = N
A
= n( A)
Hence,
P( A) =
N
A
N
=
n( A)
n( S)
=
(
30
10
)(
50
0
)
(
80
10
)
= 0.00001825
Ans: 0.265, and 0.00624 for b and c, respectively.
Limitation: The classical approach cannot be employed if:
it is not possible to enumerate all the possible outcomes for an experiment.
the sample points (outcomes) are not mutually independent.
each and every outcomes is not equally likely.
B. The Frequentist Approach
This is based on the relative frequencies of outcomes belonging to an event.
Definition: The probability of an event A is the proportion of outcomes favorable to A in the long run when
the experiment is repeated under same condition.
P( A) = l i m
N
N
A
N

Example 1: If records show that 60 out of 100,000 bulbs produced are defective. What is the probability of
a newly produced bulb to be defective?
P( A) = l i m
N
N
A
N
=
60
100,000
= 0.006
Limitations of frequencies approach:
If repeated trials are not possible we cannot use this approach

The limit as N approaches to infinitive has no clear meaning (How large is large??)
Generating infinite trials has cost imposition.
C. Subjective Approach
It is always based on some prior body of knowledge. Hence subjective measures of uncertainty are always
conditional on this prior knowledge.
Definition: Subjective probability is the degree of believe assigned to the occurrence of an event by a
particular individual.
Examples: - A medical Dr. assigns the probability that the patient recovers.
- Estimating the likely hood that you will earn an A grade in this course.
D. Axiomatic Approach:
Let E be a random experiment and S be a sample space associated with E. With each event A defined on S.
We can associate a real number called the probability of A satisfying the following properties called axioms
of probability or postulates of probability.
1. P( A) 0 (i.e. the probability of any event is non-negative real number)
2. P( S) = 1 (Sure event)
3. If A and B are two mutually exclusive events, the probability that one or the other occur equals the
sum of the two probabilities. i. e. P ( A B) = P ( A) + P ( B) .
4. If A
1
, A
2
, A
3
. form a sequence of pair wise mutually exclusive events (meaning they satisfy
A
A
]
= for all i ]) then
P( A
1
A
2
, A
3
. ) = P __ A
=1
] = P( A
=1

4.6. Derived theorems of probability
1. P( ) = 0 for any sample space S
2. If A is the complement of event A, then P( A
) = 1 P( A)
3. If A onJ B are any two events in S, then P( A B) = P( A) + P( B) P( A B)
4.7. Conditional Probability
It is to be recalled that up to this moment we were talking about events that are independent. In the real
world variables are known to be interrelated or dependent to one another, and so do events. Accordingly, in
most cases it seems rare to find probabilities of events that do not involve conditional probabilities.
The conditional probability will helps us to calculate the probability of events when it known that another
event has occurred. In particular, it is known that some event has already occurred, you will find the
probability of another event. Conditional probability refers to those probabilities that arise in the case of
dependent events. To introduce this important concept let us consider the following example.
E Ex xa am mp pl le e: : A box of 80 candles consists of 30 defective and 50 non-defective candles. Suppose that we
choose two candles from this box one after the other.
a) with replacement b) without replacement
Define two events A and B as follows
A=the first candle is defective B=the second candle is defective
If we are choosing with replacement, P(A) =P(B)= 30/80 =3/8 because each time the selected candle will be
replaced back and hence the composition remains the same.

However, if we are choosing without replacement, the results are not quite immediate.
P(A)=3/8 but what about P(B)=? It is clear that P(B) depends on whether A did occur or didnt occur. If A
did not occur then P(B)=30/79. On the other hand, if A did occur then on the second draw there are only 79
candles left of which 29 are defective. In this case P(B)= P(B/A)=29/79. Such types of probability are called
conditional probabilities.
D De ef fi in ni it ti io on n: : C Co on nd di it ti io on na al l P Pr ro ob ba ab bi il li it ty y: Let A and B be two events in the sample space S with P( B) > 0.
Then the probability that event A will given that event B had already occurred is called the conditional
probability of A given that B which we denote it by P( B/ A) is defined as
P P( ( A A/ / B B) ) = =
P P ( ( A A B B) )
P P( ( B B) )
; ; w w crc crc P P ( ( B B) ) > > 0 0
E Ex xa am mp pl le e: : Consider the above example when it is without replacement
It is easily observed that P P( ( A A) ) = = 3 3 / / 8 8
P( A B) = P( Bot h ar e defect i ves) =
(
30
2
)(
50
0
)
(
80
2
)
=
435
3160
=
87
632

P P(
B B/ / A A)
= =
P P(
A A B B)
P P(
A A)
= =
87/ 632
3 3 / / 8 8
= =
29 29
79 79

Remark:
1. P( A/ B) for fixed B satisfies all of the various axioms of probability.
I. 0 P( A/ B) 1
II. P( S/ B) = 1 ; where S denotes the Sample Space.
III. P( ( A
1
A
2
) / B) = P( A
1
/ B) + P( A
2
/ B) ; where A
1
onJ A
2
are mutually exclusive events
IV. If A
1
, A
2
, A
n
, are a sequence mutually exclusive events in S, then
P( _A
/ B)
n
=1
= P( A
/ B)
n
=1

2. If the possible outcomes are finitely many and equally likely, we have
P( A/ B) =
numbcr o clcmcnts o A B
Number of el ement s of B
=
n( A B)
n( B)

(I.e. Conditional probability can also be viewed as a probability law on a new universe B, because all of the
conditional probability is concentrated on B)
3. For every events A
1
onJ A
2
in S, then
P( ( A
1
A
2
) / B) = P( A
1
/ B) + P( A
2
/ B) P( ( A
1
A
2
) / B)
4. If A
1
onJ A
2
are events in S and A
1
A
2
then P( A
1
/ B) P( A
2
/ B)
5. If A is an event in S, then P( A
i
/ B) = 1 P( A/ B)
Example 1: Toss a fair coin three times successively. Let

A = { Horc coJs ton toils comc up}
B = { irst toss is o coJ}
Then compute P( A/ B) .
Example 2: For a student enrolling at freshman at certain university the probability that he/she will get
scholarship is 0.25 and the probability that he/she that he/she will graduate is 0.75. If the probability that
he/she will get scholarship and will also graduate is 0.2. What is the probability that a student who get a
scholarship will graduate?
Example 3: Let probability that a policy will be correctly formulated is 0.60 and the probability that a
policy will be correctly formulated and correctly executed is 0.54. Then fined the probability that:
a. the policy will be correctly executed given that it is correctly formulated
b. The policy will not be correctly executed given that it is correctly formulated.
Example 4: A lot consists of 20 defective and 80 non-defective items from which two items are chosen
without replacement. Events A & B are defined as
A = {the first item chosen is defective}, B = {the second item chosen is defective}
a. What is the probability that both items are defective?
b. What is the probability that the second item is defective?
4.8. Multiplication theorem, Bayes Theorem and Total probability theorem
4.8.1. Multiplication theorem of probability
Definition: If A and B are two events in S and P( A) is not zero, then the probability that A and B will both
occur is the product of the probability of A and the conditional probability of B given A. i.e. P( A B) =
P( A) P( B/ A) . Similarly, P( A B) = P( B) P( A/ B) .
Multiplication rule of probability is used to find the probability of simultaneous occurrence of events.
T Th he e a ab bo ov ve e m mu ul lt ti ip pl li ic ca at ti io on n r ru ul le e c ca an n b be e g ge en ne er ra al li iz ze ed d t to o m mo or re e t th ha an n t tw wo o e ev ve en nt ts s i in n t th he e f fo ol ll lo ow wi in ng g w wa ay y. .
L Le et t A A
1 1, , A A
2 2, , , , A A
n n b be e a a s se eq qu ue en nc ce e o of f e ev ve en nt ts s a an nd d a as ss su um me e =
=
A
n
1 i
i
. . T Th he en n
) A ., , A , /A P(A .. ) A , /A P(A ) /A P(A ) P(A ) A P(
1 - n 2 1 n 2 1 3 1 2 1
n
1 i
i
. . =
=

Example 1: suppose there is a box containing 20 fuses of which 5 are defective. If 3 fuses are selected at
random and removed successively, what is the probability that all are defective?
Example 2: Three balls are drawn at random one after the other without replacement from a box containing
3 white and 4 red balls. Find the probability that the balls alternate in color.
Solution
Let W: {white ball is drawn}

R: {red ball is drawn}
B: {the balls alternate in color}
B= (W R W) (R W R)
P(B)= P(W R W) + P(R W R)
= P(W)P(R/W) P(W/(WR)) + P(R) P(W/R) P(R/(R W))
= (3/7)(4/6)(1/5) + (4/7)(3/6)(3/5)
= 2/7
Example 3: Suppose that our sample space is the population of adults in a small town who have completed
the requirements for a college degree. They are categorized according to sex and employment status as
follows.
Employed(E) Unemployed Total
Male(M) 30 40 70
Female 20 70 90
Total 50 110 160
A person is selected at random. What is the probability that the selected person is
a) a man given that he is employed b)unemployed given that she is female
c) a male or a female d)unemployed male e)employed
4.8.2. Bayes Theorem
D De ef fi in ni it ti io on n ( (P Pa ar rt ti it ti io on n o of f S Sa am mp pl le e S Sp pa ac ce e) ): : W We e s sa ay y t th ha at t t th he e e ev ve en nt ts s B B
1 1, , B B
2 2, , . .B B
k k r re ep pr re es se en nt ts s p pa ar rt ti it ti io on n o of f t th he e
s sa am mp pl le e s sp pa ac ce e S S i if f
a a) ) B B
i i B B
j j = = i i. .e e. . i if f t th he e e ev ve en nt ts s a ar re e m mu ut tu ua al ll ly y e ex xc cl lu us si iv ve e
b b) ) S B
k
1 i
i
=
=

c c) ) P P( (B B
i i) ) > > 0 0 i i
Thus the events B B
1 1, , B B
2 2, , . .B B
k k a ar re e m mu ut tu ua al ll ly y e ex xc cl lu us si iv ve e a an nd d e ex xh ha au us st ti iv ve e. . G Gr ra ap ph hi ic ca al ll ly y: :


Example: Let S = { 1, 2, 3, 4, 5, 6} . B
1
= { 1, 2} , B
2
= { 3, 4, 5} , B
31
= { 6}
L Le et t A A b be e s so om me e e ev ve en nt t a as ss so oc ci ia at te ed d w wi it th h S S a an nd d l le et t B B
1 1, , B B
2 2, , B B
k k b be e p pa ar rt ti it ti io on ns s o of f S S. . H He en nc ce e w we e c ca an n write
A = (A B
1
) (A B
2
) (A B
k
)
Of course some of A B
i
may be empty. Note that since Bis are partitions and have no intersection
(mutually exclusive events) all sets (A B
1
), (A B
2
), ,(A B
k
) are pair wise mutually exclusive.
Hence P(A) = P(A B
1
) + P(A B
2
) + + P(A B
k
)
But P(A B
i
), each for each and every i, may be expressed as P(A/B
i
) P(B
i
) and we obtain what is called
the theorem of total probability.
T To ot ta al l P Pr ro ob ba ab bi il li it ty y T Th he eo or re em m
If we have B
1
, B
2
, .B
k
to be partitions of S then for any event A
P(A) = P(A/B
1
) P(B
1
) + P(A/B
2
) P(B
2
) + . + P(A/B
k
) P(B
k
)
= =
=
k
1 i
i i
) P(B ) P(A/B
G Gr ra ap ph hi ic ca al ll ly y: :

Example: A coin, is weighted so that P( H) =
2
3
and P( T) =
1
3
, is tossed. If heads appears, then a number is
selected at random from the numbers 1 through 9; if tails appears, then a number is selected at random from
the numbers 1 through 5. Find the probability that an even number is selected.
E Ex xa am mp pl le e: Three machines A, B and C produce 50%, 30% and 20% of the total output respectively. The
percentages of defective output of these machines are 3%, 4% and 5% respectively. If an item is selected at
random, what is the probability that the item is defective?
S So ol lu ut ti io on n: :
L Le et t A A: : T Th he e s se el le ec ct te ed d i it te em m i is s d de ef fe ec ct ti iv ve e
B B
1 1: : O Ou ut tp pu ut ts s p pr ro od du uc ce ed d b by y A A P P( (B B
1 1) ) = = 0 0. .5 5 P P( (A A/ /B B
1 1) ) = = 0 0. .0 03 3
B B
2 2: : O Ou ut tp pu ut ts s p pr ro od du uc ce ed d b by y B B P P( (B B
2 2) ) = = 0 0. .3 3 P P( (A A/ /B B
2 2) ) = = 0 0. .0 04 4
B B
3 3: : O Ou ut tp pu ut ts s p pr ro od du uc ce ed d b by y C C P P( (B B
3 3) ) = = 0 0. .2 2 P P( (A A/ /B B
3 3) ) = = 0 0. .0 05 5
P P( (A A) ) = = P P( (A A/ /B B
1 1) ) P P( (B B
1 1) ) + + P P( (A A/ /B B
2 2) ) P P( (B B
2 2) ) + + P P( (A A/ /B B
3 3) ) P P( (B B
3 3) )
~

= = ( (0 0. .5 5) )( (0 0. .0 03 3) )+ +( (0 0. .3 3) )( (0 0. .0 04 4) )+ +( (0 0. .2 2) )( (0 0. .0 05 5) )
= = 0 0. .0 03 37 7
T Th he e a ab bo ov ve e e ex xa am mp pl le e m ma ay y m mo ot ti iv va at te e u us s t to o a as sk k o on ne e q qu ue es st ti io on n. . S Su up pp po os se e o on ne e i it te em m i is s c ch ho os se en n a an nd d i is s f fo ou un nd d t to o b be e
d de ef fe ec ct ti iv ve e. . W Wh ha at t i is s t th he e p pr ro ob ba ab bi il li it ty y t th ha at t i it t i is s f fr ro om m m ma ac ch hi in ne e A A? ? T Th he e f fo ol ll lo ow wi in ng g w we el ll l- -k kn no ow wn n t th he eo or re em m, , c ca al ll le ed d
B Ba ay ye es s T Th he eo or re em m, , p pr ro ov vi id de es s t th he e a an ns sw we er r! !
B Ba ay ye es s T Th he eo or re em m: : L Le et t B B
1 1, , B B
2 2, , . ., , B B
k k b be e a a p pa ar rt ti it ti io on n o of f t th he e s sa am mp pl le e s sp pa ac ce e S S a an nd d l le et t A A b be e a an n e ev ve en nt t a as ss so oc ci ia at te ed d
w wi it th h S S. . T Th he en n
=
=
k
i
i i
i i
i
B P B A P
B P B A P
A B P
1
) ( ) / (
) ( ) / (
) / (

P Pr ro oo of f
=
=
=
=
=
=
k
i
i i
i i
i
k
i
i i
i i i
i
B P B A P
B P B A P
A B P
thus B P B A P
A P
B P B A P
A P
A B P
A B P
1
1
) ( ) / (
) ( ) / (
) / (
, ) ( ) / ( P(A) have we y theorem probabilit total from But then
) (
) ( ) / (
) (
) (
) / (

E Ex xa am mp pl le e: : C Co on ns si id de er r t th he e p pr re ev vi io ou us s e ex xa am mp pl le e. . A An n i it te em m w wa as s s se el le ec ct te ed d a an nd d i is s f fo ou un nd d t to o b be e d de ef fe ec ct ti iv ve e. . W Wh ha at t i is s t th he e
p pr ro ob ba ab bi il li it ty y t th ha at t i it t i is s f fr ro om m B B
1 1? ?
P P( (B B
1 1) ) = = 0 0. .5 5 P P( (A A/ /B B
1 1) ) = = 0 0. .0 03 3
P P( (B B
2 2) ) = = 0 0. .3 3 P P( (A A/ /B B
2 2) ) = = 0 0. .0 04 4
P P( (B B
3 3) ) = = 0 0. .2 2 P P( (A A/ /B B
3 3) ) = = 0 0. .0 05 5
37
15
037 . 0
) 5 . 0 )( 03 . 0 (

) ( ) / ( ) ( ) / ( ) ( ) / (
) ( ) / (
) ( ) / (
) ( ) / (
) / (
3 3 2 2 1 1
1 1
3
1
1 1
1
= =
+ +
= =
=
B P B A P B P B A P B P B A P
B P B A P
B P B A P
B P B A P
A B P
i
i i

I In n o ot th he er r w wo or rd ds s, , w we e d di iv vi id de e t th he e p pr ro ob ba ab bi il li it ty y o of f t th he e r re eq qu ui ir re ed d p pa at th h, , t th ho os se e p pa at th hs s w wh hi ic ch h l le ea ad d t to o a a d de ef fe ec ct ti iv ve e i it te em m, , b by y
t th he e p pr ro ob ba ab bi il li it ty y o of f t th he e r re ed du uc ce ed d s sa am mp pl le e s sp pa ac ce e. .
4 4. .9 9. . I In nd de ep pe en nd de en nt t e ev ve en nt ts s: :
D De ef fi in ni it ti io on n: : T Tw wo o e ev ve en nt ts s A A a an nd d B B a ar re e s sa ai id d t to o b be e i in nd de ep pe en nd de en nt t i if ff f
P P( ( A A B B) ) = = P P( ( A A) ) P P( ( B B) )

V Ve er rb ba al ll ly y s sp pe ea ak ki in ng g, , A A a an nd d B B a ar re e i in nd de ep pe en nd de en nt t p pr ro ov vi id de ed d k kn no ow wl le ed dg ge e o of f t th he e o oc cc cu ur rr re en nc ce e o of f A A b by y n no o m me ea an ns s
i in nf fl lu ue en nc ce es s t th he e p pr ro ob ba ab bi il li it ty y o of f o oc cc cu ur rr re en nc ce e o of f B B. .
R Re em ma ar rk ks s
1 1) ) I If f A A a an nd d B B a ar re e m mu ut tu ua al ll ly y e ex xc cl lu us si iv ve e w wh he en n d do o t th he ey y b be ec co om me e i in nd de ep pe en nd de en nt t? ?
P P( (A A B B) ) = = P P( (| |) ) P P( (A A) ) P P( (B B) ) = = 0 0
A A a an nd d B B a ar re e i in nd de ep pe en nd de en nt t a an nd d m mu ut tu ua al ll ly y e ex xc cl lu us si iv ve e i if ff f A A B B = = | | a an nd d e ei it th he er r P P( (A A) ) = =0 0 o or r P P( (B B) )= =0 0
2 2) ) I If f A A _ _ B B t th he en n A A B B = = A A
P P( (A A B B) ) = = P P( (A A) ) A A a an nd d B B c ca an nn no ot t b be e i in nd de ep pe en nd de en nt t u un nl le es ss s P P( (B B) )= = 1 1
B B = = S S
3 3) ) | | a an nd d S S a ar re e i in nd de ep pe en nd de en nt t o of f a an ny y e ev ve en nt t
4 4) ) T Th hr re ee e e ev ve en nt ts s A A, , B B, , a an nd d C C a ar re e j jo oi in nt tl ly y i in nd de ep pe en nd de en nt t i if ff f
i. P( A B) = P( A) P( B)
ii. P( A C) = P( A) P( C)
iii. P( B C) = P( B) P( C)
iv. P( A B C) = P( A) P( B) P( C)
It is worth noting that three events can be pair wise independent in the sense of definition of
independency of two events but not jointly independent. This is to mean that the pairwise
independency is not a sufficient condition for three events to be mutually independent. In general,
for n events to be jointly independent 2
n
n 1 independence conditions have to be satisfied.
T Th he eo or re em m: : L Le et t A A a an nd d B B b be e i in nd de ep pe en nd de en nt t. . T Th he en n
I I. . A A a an nd d B B
/ /
a ar re e i in nd de ep pe en nd de en nt t
I II I. . A A
/ /
a an nd d B B
/ /
a ar re e i in nd de ep pe en nd de en nt t
III. A A
/ /
a an nd d B B a ar re e i in nd de ep pe en nd de en nt t


CHAPTER FIVE
5. ONE DIMENSIONAL RANDOM VARIABLES
5.1 Random variable: definition and distribution function
5.2 Discrete random variables
5.3 Continuous random variables
5.4 Cumulative distribution function and its properties
5.5 Functions of Random Variables
5.5.1. Equivalent events
5.5.2. Functions of discrete random variables and their distributions
5.5.3. Functions of continuous random variables and their distributions
5.6 Expectation and Variance of a random variable, function of a random variable and their Properties
5.7 Moments and moment generating function
5.1 Random variable
Introduction
The theory of probability distributions is widely used in both natural and social science. For that matter, the
concept of random variables and distributions is the backbone of statistics. Particularly, the area of
inferential statistics is entirely dependent on these concepts. In estimation and hypotheses testing we
extensively use some commonly known distributions.
Definition: Let E be an experiment and S be a sample space associated with the experiment. A function X
that assigning to every element s S , a real number, X( s) , is called a random variable. The set of all
possible values of the random variable X is called the range space. Moreover, the domain of the random
variable X is S.
Random variables are nothing but real valued functions that transform events to real numbers. Hence,
one can say that random variables serve as links between events and real numbers. Their significance
can be seen from the fact that it is theoretically easier to deal with numbers than events.
A random variable, X, is a function that assigns a single, but variable, value to each element of a sample
space.
A random variable, X, provides a means of assigning numerical values to experimental outcomes.
A random variable, X, is a numerical description of the outcomes of the experiment or a numerical
valued function defined on sample space, usually denoted by capital letters.

Example 1: Consider the different possible orderings of boy (B) and girl (G) in four sequential births.
There are 2*2*2*2=24 = 16 possibilities, so the sample space is:
BBBB GGBB BGBB GBBB
BBBG BGBG GBBG GGBG
BBGB BGGB GBGB GGGB
BBGG BGGG GBGG GGGG

If girl and boy are each equally likely [P(G) = P(B) = 1/2], and the gender of each child is independent of
that of the previous child, then the probability of each of these 16 possibilities is:
(1/2)(1/2)(1/2)(1/2) = 1/16.
Now count the number of girls in each set of four sequential births:
BBBB (0) GGBB (2) BGBB (1) GBBB (1)
BBBG (1) BGBG (2) GBBG (2) GGBG (3)
BBGB (1) BGGB (2) GBGB (2) GGGB (3)
BBGG (2) BGGG (3) GBGG (3) GGGG (4)
Notice that:
each possible outcome is assigned a single numeric value,
all outcomes are assigned a numeric value, and
the value assigned varies over the outcomes.
The count of the number of girls is a random variable:

Where R
X
is the range space of X and S is the sample space.
Example 2: Consider the experiment of tossing of fair coin once.
The sample space is S= {H, T} where H denotes the outcome Head and T denotes the outcome Tail. So,
there are two possible outcomes H or T.
Now, let the random variable X represents the outcome `Head, then X can take the value 0 or 1.
Example 3: Suppose a single fair die is rolled once.

The sample space of this experiment constitutes six possible outcomes, S = {1, 2, 3, 4, 5, 6}
Let the random variable X denotes the outcome A number greater than 2 occurs. Then the random variable
can assume the values 3, 4, 5 or 6.
Probability Distribution
The probability distribution for a random variable describes how the probabilities are distributed over the
values of the random variable
Types of Random Variables
Depending upon the numerical values it can assume, a random variable can be classified into two major
divisions.
Discrete Random Variable and
Continuous Random Variable
5.2. Discrete Random Variable
Definition: Let X be a random variable. If the number of possible values of X (i.e. Rx) is finite or countably
infinite, then we call X is a discrete random variable. That is, the possible values of X may be listed as
x
1
, x
2
, , x
n
, In the finite case the list terminates and in the countably infinite case the list continues
indefinitely.
has a countable number of possible values
has discrete jumps (or gaps) between successive values
has measurable probability associated with individual values
counts
Examples:
Toss a coin n times and count the number of heads.
Number of children in a family.
Number of car accidents per week.
Number of defective items in a given company.
Number of bacteria per two cubic centimeter of water.
Probability distribution of discrete random variable
Definition: : Let X be a discrete random variable. Hence, R
X
consists of at most a countably infinite number
of values, x
1
, x
2
, with each possible outcome x
we associate a number P( x
) = P( X = x
) called the
probability of x
.The probabilities P( x
) for i = 1,2,3, . . . , n, . .. must satisfy the following two conditions:

I. P( x
) 0 for all i,
II. P( x
=1
= 1
The function p defined above is called the probability function (or point probability function) of the
random variable X. Furthermore, the tabular arrangement of all possible values of X with their
corresponding probability is called probability distribution of X.

Example 1: Construct a probability distribution for the number of heads in tossing of a fair coin two times.
Solution: The sample space of the experiment contains the following:
S = {HH, HT, TH, TT}
Let the random variable X denotes the number of heads. We then use the probability function P(X) to
assign probability to each outcome consequently; the probability distribution is given below:
x
0 1 2
P( x
) 1/4 1/2 1/4

Example 2: The number of mistakes a typist made in ten days of assessment is shown in the following
table.
Number of mistakes 2 3 4 5
Number of days 1 4 3 2

Construct a probability distribution for the number of mistakes she committed.
Solution: Let the variable X denotes the number of mistakes the typist committed. Then, we assign a
probability for each of the number of days with respect to the total number of days.
The probability distribution is shown below:
Number of mistakes (x
i
) 2 3 4 5
P( x
) 1/10 4/10 3/10 2/10

Notes:
a) Suppose that the discrete random variable X may assume only a finite number of values, say
x
1
, x
2
, , x
n
. If each outcome is equally probable, then we obviously have P( x
1
) = P( x
2
) = =
P( x
n
) =
1
n
.
b) If X assumes a countably infinite number of values, then it is impossible to have all outcomes
equally probable. For we cannot possibly satisfy the condition P( x
I
)
I=1
= 1 if we must have
P( x
) = c for all i.
c) In every finite interval there will be at most a finite number of possible values of X. if some such
interval contains name of these possible values we assign probability zero to it. That is, if R
X
=
{ x
1
, x
2
, , x
n
} and if no x
[ a, b] , then P( o X b) = 0.
5.3. Continuous Random Variable
Definition: A random variable X is said to be a continuous random variable if it assumes all values in some
interval (c,d), where c and d are real numbers.

has an uncountably infinite number of possible values
moves continuously from value to value
has no measurable probability associated with each value
Examples:
Height of students at certain college.
Mark of a student.
Life time of light bulbs.
Length of time required to complete a given training.
Probability density function ( pdI) of continuous random variable
A random variable X is said to be a continuous random variable if there exists a function , called the
probability density function ( pdf ) of X, satisfying the following conditions:
a) ( x) 0 for all x
b) ( x) Jx
-
= 1
c) For any o onJ b wit < o < b < , wc o:c P( o X b) = ( x) Jx
b
u

Remark:
a. P(axb) represents the area under the graph ( or curve) of the P.d.f of f between a and b.

b b. . It is a consequence of the above description of X that for any specified value of X, say X
0
,
w we e h ha av ve e p p( (X X= =X X
0 0) )= =0 0, , s si in nc ce e P P( (X X= =X X
0 0) )= = 0 ) (
0
0
=
}
dx x f
x
x

W We e m mu us st t r re ea al li iz ze e, , h ho ow we ev ve er r, , t th ha at t i if f w we e a al ll lo ow w X X t to o a as ss su um me e a al ll l t th he e v va al lu ue es s i in n s so om me e i in nt te er rv va al l, , t th he en n t th he e z ze er ro o v va al lu ue e
o of f t th he e p pr ro ob ba ab bi il li it ty y d do oe es s n no ot t i im mp pl ly y t th ha at t t th he e e ev ve en nt t i is s i im mp po os ss si ib bl le e. . T Th ha at t i is s, , z ze er ro o p pr ro ob ba ab bi il li it ty y d do oe es s n no ot t n ne ec ce es ss sa ar ri il ly y
i im mp pl ly y i im mp po os ss si ib bi il li it ty y. . H He en nc ce e, , i in n t th he e c co on nt ti in nu uo ou us s c ca as se e P P( (A A) )= =0 0 d do oe es s n no ot t i im mp pl ly y A A= =| |. . I In n v vi ie ew w o of f t th hi is s f fa ac ct t, , t th he e
f fo ol ll lo ow wi in ng g p pr ro ob ba ab bi il li it ti ie es s a ar re e a al ll l t th he e s sa am me e i if f X X i is s a a c co on nt ti in nu uo ou us s r ra an nd do om m v va ar ri ia ab bl le e. .
P P( (c c X X d d) )= = P P( (c c X X< <d d) )= = P P( (c c< <X X d d) )= = P P( (c c< <X X< <d d) )
c. If X assumes values only in some interval [a, b], we may simply set f ( x) = 0 for all x [ a, b] .
E Ex xa am mp pl le e 1 1: : L Le et t X X b be e a a c co on nt ti in nu uo ou us s r ra an nd do om m v va ar ri ia ab bl le e w wi it th h p pr ro ob ba ab bi il li it ty y d de en ns si it ty y f fu un nc ct ti io on n ( (P P. .d d. .f f) )
< <
=
otherwise 0
1 0 if 2
) (
x x
x f
a a) ) V Ve er ri if fy y t th ha at t f f i is s a a P P. .d d. .f f b b) ) f fi in nd d P P( (0 0. .5 5< <x x< <0 0. .7 75 5) )

a a) ) A A f fu un nc ct ti io on n f f i is s a a P P. .d d. .f f i if ff f t th he e f fo ol ll lo ow wi in ng g t tw wo o c co on nd di it ti io on ns s a ar re e s sa at ti is sf fi ie ed d
i i) ) f f( (x x) )> >0 0 a an nd d i ii i) ) 1 ) ( =
}
+

dx x f
I In n o ou ur r c ca as se e, , i it t i is s e ea as si il ly y o ob bs se er rv va ab bl le e t th ha at t f f ( (x x) ) > > 0 0 b be ec ca au us se e 2 2x x i is s a a p po os si it ti iv ve e n nu um mb be er r f fo or r a al ll l x x b be et tw we ee en n 0 0 a an nd d
1 1. .M Mo or re eo ov ve er r, , f fo or r o ot th he er r v va al lu ue es s o of f x x t th he e g gi iv ve en n f fu un nc ct ti io on n a as ss su um me es s t th he e v va al lu ue e 0 0. . H He en nc ce e t th he e f fi ir rs st t c co on nd di it ti io on n i is s
s sa at ti is sf fi ie ed d. . W We e c ca an n a al ls so o s sh ho ow w t th he e s se ec co on nd d c co on nd di it ti io on n a as s u un nd de er r. .
1 0 1 0 0dx 2 0 f(x)dx ) ( ) ( ) (
1
0 1
0 1
0 1
0
= + + = + + = + + =
} } } } } } }
+

+

+

xdx dx dx x f dx x f dx x f
T Th he er re ef fo or re e, , w we e c ca an n c co on nc cl lu ud de e t th ha at t t th he e g gi iv ve en n f fu un nc ct ti io on n i is s a a P P. .d d. .f f. .
b b) ) P P ( (0 0. .5 5< <x x< <0 0. .7 75 5) )= =
16
5
4
1
16
9
x 2
75 . 0
5 . 0
2
75 . 0
5 . 0
= = =
}
xdx
E Ex xa am mp pl le e 2 2: : S Su up pp po os se e t th ha at t X X i is s a a c co on nt ti in nu uo ou us s r ra an nd do om m v va ar ri ia ab bl le e w wi it th h P P. .d d. .f f
s s +
=
otherwise 0
3 x 0 if k x
6
1
) (x f

a a) ) E Ev va al lu ua at te e k k b b) ) P P( ( 1 1 X X 2 2 ) )
E Ex xa am mp pl le e 2 2: : S Su up pp po os se e t th ha at t X X i is s a a c co on nt ti in nu uo ou us s r ra an nd do om m v va ar ri ia ab bl le e w wi it th h P P. .d d. .f f
s s
=
otherwise 0
1 x 0 if 3x
) (
2
x f
a a) ) V Ve er ri if fy y t th ha at t f f i is s a a P P. .d d. .f f b b) ) P P( (X X> >1 1/ /3 3) ) c c) ) P P( (1 1/ /4 4 s s x x s s 2 2/ /3 3) )
C Co on nd di it ti io on na al l P Pr ro ob ba ab bi il li it ty y i in n t th he e C Ca as se e o of f C Co on nt ti in nu uo ou us s R Ra an nd do om m V Va ar ri ia ab bl le es s
R Re ec ca al ll l t th ha at t f fo or r t th he e c ca as se e o of f d di is sc cr re et te e r ra an nd do om m v va ar ri ia ab bl le es s t th he e c co on nd di it ti io on na al l p pr ro ob ba ab bi il li it ty y i is s g gi iv ve en n b by y
) (
) (
) / (
B P
B A P
B A P

=
T Th he e s sa am me e f fo or rm mu ul la at ti io on n h ho ol ld ds s t tr ru ue e f fo or r t th he e c ca as se e o of f c co on nt ti in nu uo ou us s r ra an nd do om m v va ar ri ia ab bl le es s e ex xc ce ep pt t t th ha at t A A a an nd d B B t ta ak ke e r ra an ng ge e
o of f v va al lu ue es s ( (i in nt te er rv va al ls s) ) r ra at th he er r t th ha an n d di is sc cr re et te e v va al lu ue es s. .
E Ex xa am mp pl le e: : T Th he e d di ia am me et te er r o of f a an n e el le ec ct tr ri ic c c ca ab bl le e i is s a as ss su um me ed d t to o b be e a a c co on nt ti in nu uo ou us s r ra an nd do om m v va ar ri ia ab bl le e, , s sa ay y X X, , w wi it th h
P P. .d d. .f f
s s
=
otherwise 0
1 0 if ) 1 ( 6
) (
x x x
x f
C Co om mp pu ut te e
|
|
.
|
\
|
< < s
3
2
3
1
2
1
x x P
~

2 / 1
) 1 ( 6
) 1 ( 6
3
2
3
1
2
1
3
1
3
2
3
1
)
3
2
3
1
( )
2
1
(
3
2
3
1
2
1
3 / 2
3 / 1
2 / 1
3 / 1
=
=
|
.
|
\
|
< <
|
.
|
\
|
< <
=
|
.
|
\
|
< <
|
.
|
\
|
< < s
=
|
|
.
|
\
|
< < s
}
}
dx x x
dx x x
x P
x P
x P
x x P
x x P

5.4 Cumulative distribution function and its properties
D De ef fi in ni it ti io on n ( (C Cu um mu ul la at ti iv ve e D Di is st tr ri ib bu ut ti io on n F Fu un nc ct ti io on n) ) L Le et t X X b be e a a r ra an nd do om m v va ar ri ia ab bl le e ( (d di is sc cr re et te e o or r c co on nt ti in nu uo ou us s) ). . T Th he e
f fu un nc ct ti io on n F F t th ha at t i is s d de ef fi in ne ed d a as s
F F( ( x x) ) = = P P( ( X X x x) )
i is s c ca al ll le ed d t th he e c cu um mu ul la at ti iv ve e d di is st tr ri ib bu ut ti io on n f fu un nc ct ti io on n ( (C Cd df f) ) o of f t th he e r ra an nd do om m v va ar ri ia ab bl le e X X. .
N No ot ta at ti io on n: : C Cu um mu ul la at ti iv ve e d di is st tr ri ib bu ut ti io on n f fu un nc ct ti io on n i is s u us su ua al ll ly y d de en no ot te ed d b by y u up pp pe er r c ca as se e a al lp ph ha ab be et ts s. .
T Th he eo or re em ms s
1 1. . I If f X X i is s a a d di is sc cr re et te e r ra an nd do om m v va ar ri ia ab bl le e, , t th he en n
x x p x F
j
j
s =
j
x all for ) ( ) (
2 2. . I If f X X i is s a a c co on nt ti in nu uo ou us s r ra an nd do om m v va ar ri ia ab bl le e, , t th he en n

}
+

= ds s f x F ) ( ) (
E Ex xa am mp pl le e 1 1: : L Le et t S S= ={ {H H, ,T T} } a an nd d X X: : t th he e n nu um mb be er r o of f h he ea ad ds s. .
C Cl le ea ar rl ly y, , X X i is s a a d di is sc cr re et te e r ra an nd do om m v va ar ri ia ab bl le e. . S Si in nc ce e P P( (x x= =0 0) )= =1 1/ /2 2= =P P( (x x= =1 1) ), , w we e h ha av ve e

>
< s
<
=
1 x if 1
1 x 0 if 2 / 1
0 X if 0
) (x F

G Gr ra ap ph hi ic ca al ll ly y, ,

E Ex xa am mp pl le e 2 2: : S Su up pp po os se e t th ha at t a a r ra an nd do om m v va ar ri ia ab bl le e X X a as ss su um me es s t th he e s si ix x v va al lu ue es s 1 1, , 2 2, , 3 3, , 4 4, , 5 5, , 6 6 w wi it th h p pr ro ob ba ab bi il li it ty y 1 1/ /6 6. .
T Th he en n, , c co om mp pu ut te e F F( (x x) )
E Ex xa am mp pl le e 3 3: : F Fi in nd d t th he e c cd df f o of f t th he e r ra an nd do om m v va ar ri ia ab bl le e X X w wh ho os se e P P. .d d. .f f i is s g gi iv ve en n b by y

s s
< <
=
Otherwise 0
2 x 1 if 2
1 x 0 if x
) ( x x f
} } } } }
+

+ + + = = s =
2
2
1
1
0
0
) ( ) ( ) ( ) ( ) ( ) ( F(x) dt t f dt t f dt t f dt t f dt t f x X P
x

C Ca as se e 1 1: : F Fo or r x xs s0 0
0 0 ) ( F(x)
} }

= = =
x x
dt dt t f
C Ca as se e 2 2: : F Fo or r 0 0s s x x s s 1 1
2 2
t
0 ) ( ) ( ) ( ) ( F(x)
2
0
2
0 0
0
x
tdt dt t f dt t f dt t f x X P
x
x x x
= = + = + = = s =
} } } }

C Ca as se e 3 3: : F Fo or r 1 1s s x x s s 2 2
1
2
2
2
2
2
1

2
2
t
0 ) ( ) ( ) ( ) ( ) ( F(x)
2
1
2
1
1
0
2 1
0 1
0
=
|
|
.
|
\
|
+ =
+ + = + + = = s =
} } } } }

x
x
x
x
x dt t f dt t f dt t f dt t f x X P
x
x x x

C Ca as se e 4 4: : F Fo or r x x> >2 2
1 0
2
3
2
2
1
0
2
2
2
t
0
) ( ) ( ) ( ) ( ) ( ) ( F(x)
2
2
1
2
1
0
2
2
1
0
2
1
0
= + |
.
|
\
|
+ = +
|
|
.
|
\
|
+ + =
+ + + = = s =
}
} } } } }

x
x x
dt
x
x
dt t f dt t f dt t f dt t f dt t f x X P

T Th he er re ef fo or re e, , t th he e c cd df f o of f X X b be ec co om me es s, ,
>
< s
< s
<
=
2 x if 1
2 x 1 if 1 -
2
x
- 2x
1 x 0 if
2
0 x if 0
) (
2
2
x
x F


E Ex xe er rc ci is se es s: :
1 1. . F Fi in nd d t th he e c cd df f o of f a a r ra an nd do om m v va ar ri ia ab bl le e X X w wh ho os se e P P. .d d. .f f i is s g gi iv ve en n b by y
>
=
Otherwise 0
0 x if e
) (
-x
x f
2 2. . D De et te er rm mi in ne e t th he e c cd df f o of f t th he e r ra an nd do om m v va ar ri ia ab bl le e X X w wi it th h P P. .d d. .f f
s s +
< s
< s
=
Otherwise 0
3 2 if
2
3
2
1
-
2 x 1 if
2
1
1 x 0 if x
2
1
) (
x x
x f

P Pr ro op pe er rt ti ie es s o of f C Cu um mu ul la at ti iv ve e D Di is st tr ri ib bu ut ti io on n F Fu un nc ct ti io on n ( (c cd df f) ) F F
a a) ) T Th he e f fu un nc ct ti io on n F F i is s a a n no on n- -d de ec cr re ea as si in ng g f fu un nc ct ti io on n, ,
b b) ) 1 ) ( lim and 0 ) ( lim
x x
= =
+
x F x F
T Th he eo or re em m
I If f X X i is s a a c co on nt ti in nu uo ou us s r ra an nd do om m v va ar ri ia ab bl le e, , t th he en n w we e h ha av ve e ) ( ) ( x F
dx
d
x f = , , f fo or r a al ll l x x a at t w wh hi ic ch h F F i is s d di if ff fe er re en nt ti ia ab bl le e. .
O On n t th he e o ot th he er r h ha an nd d, , i if f X X i is s a a d di is sc cr re et te e r ra an nd do om m v va ar ri ia ab bl le e w wi it th h p po os ss si ib bl le e v va al lu ue es s x x
1 1
, , x x
2 2
, , a an nd d s su up pp po os se e t th ha at t i it t i is s
p po os ss si ib bl le e t to o l la ab be el l t th he es se e v va al lu ue es s s so o t th ha at t x x
1 1
< < x x
2 2
< < t th he en n w we e h ha av ve e ) ( ) ( ) (
1
=
j j j
x F x F x P . .
P Pr ro oo of f
C Ca as se e 1 1: : X X i is s a a c co on nt ti in nu uo ou us s r ra an nd do om m v va ar ri ia ab bl le e
calculus of theorem l fundamenta the .......by ) ( ) (
dx
d
) (
dx
d

) ( ) (
x f dt t f x F
dt t f x F
x
x
= =
=
}
}

C Ca as se e 2 2: : X X i is s a a d di is sc cr re et te e r ra an nd do om m v va ar ri ia ab bl le e
) ( ) ( ) ( ) (
) ( ) ( ... ) ( ) ( ) ( ) (
) ( ) ( ... ) ( ) ( ) ( ) (
1
1 2 2 1 1 1
1 2 1
x f x P x F x F
x P x P x P x P x x P x F
x P x P x P x P x x P x F
j j j
j j j j
j j j j
= =
+ + + + = s =
+ + + + = s =

R Re em ma ar rk k: : P P( (a as s X X s s b b) )= =F F( (b b) )- -F F( (a a) )
E Ex xa am mp pl le e: : S Su up pp po os se e X X i is s a a c co on nt ti in nu uo ou us s r ra an nd do om m v va ar ri ia ab bl le e w wi it th h c cd df f
>
s
=
0 x if e - 1
0 x if 0
) (
x -
x F

F Fi in nd d t th he e P P. .d d. .f f o of f X X. .
5.5 Functions of Random Variables
Introduction:
Suppose we know that the distribution of a random variable X. In addition to knowing the distribution of the
random variable X we may also be interested to know the distribution of another random variable, say Y,
which is a function of the original random variable X. That is, given the distribution of the random variable
X we want to find or determine the distribution of the function Y=H(X). The transformation process is
presented in the following diagram.

Note that Y is a real valued function of the random variable X with domain Rx and range Ry. Moreover, the
function Y itself is a random variable.
Thus, let E be an experiment, S be a sample space associated with E and X be a random variable defined on
S. Suppose that y = H( x) is a real valued function of x. then Y = H( X) is a random variable since for
every s S, a value of Y is determined, say y = H( X( s) ) . As before we call Rx the range space of X, the set
of all possible values of the function X. similarly we define RY the range space of Y, the set of all possible
values of the function Y.
I Il ll lu us st tr ra at ti iv ve e E Ex xa am mp pl le e: : R R
x x
Let X be a random variable that represents the radius of a circle. Consider another variable Y that represents
the area of a circle. Clearly, Y is a function of X, and their functional relationship is presented as Y=HX
2
.
5.5.1. Equivalent events
D De ef fi in ni it ti io on n: Let C be an event associated with the range space of Y, R
Y
, i.e. CR
. Moreover, let BR
X
be
defined as B = { xeR
X
H( x) eC} . Then B and C are equivalent events. Meaning, each and every element in
C is a function of a corresponding element in B.
Note that:
The informal interpretation of the above is that B and C are equivalent events if and only if B and C
occur together. That is, when B occurs, C occurs and conversely.
When we speak of equivalent events (in the above sense), these events are associated with different
sample space.
Example: Suppose that E( x) = nx
2
. Then the events B = { X > 2} onJ C = { > 4n} are equivalent. Thus,
for if = nX
2
, then { X > 2} occurs if and only if { > 4n} occurs

Definition: Suppose X is a random variable defined on the sample space S and Rx be its range space. Let
Y=H(X) be a random variable with range space Ry where H is a real valued function. For any event CR
,
P( C) is defined as
P( C) = { xeR
X
H( x) eC}
In words: The probability of an event associated with the range space of Y is defined as the probability of
the equivalent event (in terms of X).
In other words, we can evaluate probabilities involving events associated with the function of the random
variable X if the P.d.f of X is known and if it is possible to determine the equivalent event.
E Ex xa am mp pl le e1 1: : L Le et t X X r re ep pr re es se en nt t t th he e r ra ad di ii i o of f c ci ir rc cl le es s. . S Su up pp po os se e t th ha at t H H( (x x) )= =2 2H Hx x. .
B B
1 1= ={ {x x> >2 2} } i is s e eq qu ui iv va al le en nt t t to o C C
1 1= ={ {y y> >4 4H H} }
B B
2 2= ={ {x xs s5 5} } i is s e eq qu ui iv va al le en nt t t to o C C
2 2= ={ {y ys s1 10 0H H} }
H He en nc ce e, , P P( (B B
1 1) )= =P P( (C C
1 1) ) a an nd d P P( (B B
2 2) )= =P P( (C C
2 2) )
E Ex xa am mp pl le e 2 2: : L Le et t X X b be e a a c co on nt ti in nu uo ou us s r ra an nd do om m v va ar ri ia ab bl le e w wi it th h P P. .d d. .f f
>
=
therwise 0
0 x if e
) (
-x
O
x f
S Su up pp po os se e t th ha at t a a r ra an nd do om m v va ar ri ia ab bl le e Y Y i is s d de ef fi in ne ed d b by y
2
1 +
=
x
Y . . I If f C C = = { { y y: : y y 7 7 } } , , f fi in nd d P P( (C C) ). .
I It t c ca an n b be e s se ee en n t th ha at t R R
X X= = { { x x: : x x > > 0 0 } } a an nd d R R
Y Y= = { { y y: : y y > > 1 1 / / 2 2 } } , , w wh hi ic ch h i im mp pl li ie es s t th ha at t C C_ _R R
Y Y. .
15 x 7
2
1 x
7 > >
+
> y
T Th hu us s, , d de ef fi in ne e B B a as s B B= ={ {X X: : X X > > 1 15 5} } s so o t th ha at t B B a an nd d C C a ar re e e eq qu ui iv va al le en nt t. .
H He en nc ce e, ,
15
15
1
P(B) P(C)
e
dx e
x
= = =
}
+

E Ex xe er rc ci is se e: : S Su up pp po os se e t th ha at t t th he e l le en ng gt th h o of f t ti im me e i it t t ta ak ke es s E Ea ar rl l t to o d dr ri iv ve e w wo or rk k e ev ve er ry yd da ay y i is s a a r ra an nd do om m v va ar ri ia ab bl le e X X
( (m me ea as su ur re ed d i in n h ho ou ur rs s) ) w wi it th h d di is st tr ri ib bu ut ti io on n f fu un nc ct ti io on n, ,
s s
=
therwise 0
4
3
x
4
1
if )
4
1
- 8(x
) (
O
x f
I If f w we e d de ef fi in ne e a a r ra an nd do om m v va ar ri ia ab bl le e Y Y a as s Y Y= =6 60 0X X, , t th he en n f fi in nd d P P( (Y Y< <4 45 5) ). .
R Re em ma ar rk k: : T Th he e o ot th he er r p pr ro oc ce ed du ur re e f fo or r f fi in nd di in ng g p pr ro ob ba ab bi il li it ti ie es s a as ss so oc ci ia at te ed d w wi it th h Y Y i is s t to o f fi in nd d P P. .d d. .f f o of f Y Y f fr ro om m t th he e
k kn no ow wn n P P. .d d. .f f o of f X X. .

5.5.2. Functions of discrete random variables and their distributions
C Ca as se e I I: : X X i is s a a d di is sc cr re et te e r ra an nd do om m v va ar ri ia ab bl le e: :
I If f X X i is s d di is sc cr re et te e r ra an nd do om m v va ar ri ia ab bl le e a an nd d Y Y= =H H( (X X) ), , t th he en n i it t f fo ol ll lo ow ws s i im mm me ed di ia at te el ly y t th ha at t Y Y i is s a al ls so o a a d di is sc cr re et te e r ra an nd do om m
v va ar ri ia ab bl le e. . T Th ha at t i is s i if f t th he e p po os ss si ib bl le e v va al lu ue es s o of f X X a ar re e x x
1 1, ,x x
2 2, , , ,x x
n n t th he en n t th he e p po os ss si ib bl le e v va al lu ue es s o of f Y Y a ar re e
H H( (x x
1 1) ), ,H H( (x x
2 2) ), , , ,H H( (x x
n n) ). .
D De ef fi in ni it ti io on n: : If x
1
, x
2
, , x
n
, . are the possible values of X, P( x
I
) = P( X = x
I
) , and H is a function such that
to each value y there corresponds exactly one value x, then the probability distribution of Y is obtained as
follows:
Possible values of Y: y
I
= H( x
I
) , i = 1,2, , n,
Probabilities of Y: q( y
I
) = P( Y = y
I
) = P( x
I
)
Example 1: Consider the following probability distribution of a discrete random variable X
x
I
-1 0 1
P( x
I
) 1/3 1/2 1/6
Find the probability distribution of = 3X + 1.
Solution:
y
-2 1 4
q( y
) 1/3 1/2 1/6

Note that: It may happen that several values of X lead to the same value of Y. In other words, the relation
between X and Y may not necessarily be One-to-One, as the following examples illustrate.
E Ex xa am mp pl le e 1 1: Consider the above example. Moreover let Y=X
2
. Hence the possible value ofs of Y are zero
and one, assumed with probabilities and . Thus, y=1 corresponds to not only one possible value in the
range space of X, but it refers to two possible values, namely x=1 and x=-1. Accordingly,
P(Y=1)=P(x=1)+P(x=-1). For y = 1 iff X =1 or X = -1 and the probabilities of this latter event is
1
3
+
1
6
= 1
Definition: let x
1
, x
2
, , x
k
, represent the X- values having the property E(x
]
) = y
or oll ]. Then
q( y
) = P( Y = y
I
) = P( x
1
) + P( x
2
) + P( x
k
) +
E Ex xa am mp pl le e 2 2: : S Su up pp po os se e t th he e p pr ro ob ba ab bi il li it ty y f fu un nc ct ti io on n o of f a a d di is sc cr re et te e r ra an nd do om m v va ar ri ia ab bl le e X X i is s
x x - -1 1 0 0 1 1 2 2
P P( ( X X = = x x) )
7 7
30 30

8 8
30 30

4 4
30 30

11 11
30 30


F Fi in nd d t th he e p pr ro ob ba ab bi il li it ty y f fu un nc ct ti io on n o of f Y Y i if f
a a) ) Y Y= =X X
2 2
b b) )Y Y= =2 2X X+ +5 5
Example 3: Let X be a discrete random variable having the possible values 1, 2, 3, . . . , n, . .. and suppose that
P( X = x) =
1
2
x
. Find the probability distribution of Y if
= _
1 i X is c:cn
1 i X is oJJ

Case II: If X is a continuous random variable.
X may assume all real values while Y is defined to be +1 if X > 0 and -1 if X < 0. In order to obtain the
probability distribution of Y, simply determine the equivalent event (in the range space R
x
) corresponding to
the different values of Y. In the above case P( = 1) = P( X > 0) and P( = 1) = P( X 0) .
In general case, if { = y
} is equivalent to an event, say A, in the range space of X, then

q( y
) = P( = y
) = _ ( x) Jx
A

5.5.3. Functions of continuous random variables and their distributions
Here in this section our interest is to determine the Probability Density Function of a function of a
continuous random variable. Let X be a continuous random variable and Y be another random variable
which is a function of X. That is Y=H(X). Moreover, assume that the random variable Y is a continuous
random variable. To find or determine the P.d.f of Y=H(X), we follow the following steps:
a) Obtain the Cumulative Distribution Function (cdf) of Y, denoted by G(y), where 0( y) = P( < y)
in terms of the cdf or P.d.f of X by finding an event A in Rx which is equivalent to the event Ysy.
b) Differentiate G(y) with respect to y in order to obtain g(y),which is going to be the P.d.f of Y.
c) Determine those values of y in the range space of Y for which g(y)>0.
E Ex xa am mp pl le e 1 1: : S Su up pp po os se e t th ha at t X X h ha as s P P. .d d. .f f g gi iv ve en n b by y: :
< <
=
therwise 0
2 x 0 if
2
x
) (
O
x f
L Le et t H H( (X X) )= =2 2X X+ +4 4. . F Fi in nd d t th he e P P. .d d. .f f o of f Y Y= =H H( (X X) )
S St te ep p 1 1: :
}
= = =
s = s + = s =
2
4
0
2
2
4
0
2
16
) 4 (
4 2
)
2
4
( ) 4 2 ( ) ( ) (
y
y
y x
dx
x y
X P y X P y Y P y G
S St te ep p 2 2: :
8
) 4 (
16
) 4 ( 2
) ( ) (

=
= =
y y
y G
dy
d
y g

S St te ep p 3 3: : f f( (x x) )> >0 0 i if f 0 0< <x x< <2 2 w wh hi ic ch h i im mp pl li ie es s t th ha at t t th he e r ra an ng ge e s sp pa ac ce e o of f y y= =2 2x x+ +4 4 i is s. . M Mo or re eo ov ve er r, , 0
8
) 4 (
) ( >
=
y
y g i if f
y y> >4 4 t th he e i in nt te er rs se ec ct ti io on n o of f w wh hi ic ch h w wi it th h t th he e r ra an ng ge e s sp pa ac ce e o of f Y Y g gi iv ve es s 4 4< <y y< <8 8. .
T Th he er re ef fo or re e, , t th he e P P. .d d. .f f o of f Y Y i is s f fo ou un nd d t to o b be e: :
< <
=
therwise 0
8 y 4 if
8
4 - y
) (
O
y g
E Ex xa am mp pl le e 2 2: : L Le et t a a r ra an nd do om m v va ar ri ia ab bl le e X X h ha as s a a d de en ns si it ty y f fu un nc ct ti io on n
< <
=
therwise 0
1 x 0 if 1
) (
O
x f . . L Le et t H H( (X X) )= =e e
x x
. .
T Th he en n f fi in nd d t th he e P P. .d d. .f f o of f Y Y= =H H( (X X) ). .
Example 3: suppose that X has the pdf
f ( x) = ]
2x, 0 < x < 1
0 ot her w i se

Then find the pdf of E( x) = 3x + 1
Theorem: Let X be a continuous random variable with pdf , where ( x) > 0 for o < x < b. suppose that
y = E( x) is a strictly monotone (increasing or decreasing) function of x. Assume that this function is
differentiable ( and hence continuous) for all x. Then the random variable Y is defined as = E( X) has a
pdf g given by,
g( y) = ( g
-1
( x) ) _
J
Jy
g
-1
( x) _ wcrc x is cxprcsscJ intcrms o y.
Theorem: Let X be a continuous random variable with pdf and = X
2
. Then the random variable Y has
the pdf given by:
g( y) =
1
2y
|(
y) + (y)].
5.6 Expectation and Variance of a random variable, function of a random variable and their
Properties
5.6.1 Expectation of random Variable
Introduction
Most probability distributions are characterized by their mean and variance. The mean of a random variable
is referred to as expectation, provided the expected value converges to a constant. In addition to
characterizing a given probability distribution, expected values are used in further statistical analyses. For
instance, in the theory of estimation verifying whether a given estimator is an unbiased estimator or not we
need to have the concept of expectation. For that matter, calculation of variance of a given random variable
itself calls for expectation. In short, it will not be mistaken to conclude that almost all statistical analyses
are, directly or indirectly, related to expectation.
Still more, the concept of expected value is used extensively in disciplines like management. Particularly, in
decision trees we use expected values. Moreover, the expected monetary value (EMV) and other related
variables required in decision making all calls for the notion of expectation.

It is, therefore, desirable to summarize some of the outstanding features of a probability law of numerical
valued random phenomenon by specifying only a few numbers rather than entire function. Such numbers
are provided by expectation of various functions with respect to the probability law of the random
phenomenon.
5.6.1.1 Expectation of a random variable
Definition (Expectation of a discrete random Variable): Let X be a discrete random variable with
possible values x
1
, x
2
, x
3
, , x
n
. Let P( x
) = P( X = x
) i = 1,2,3 n. Then the expected

value of X, denoted by E( X) , is defined as:
E( X) = x
P( x
=1

Note: If the series converges absolutely, this number is also called the mean of X.
Example 1: A manufacturer produces items such that 10% are defective and 90% are non-defective. If a
defective item is produced the manufacturer loses $1 and if a non-defective item is produced the
manufacturer gets $5 profit. Find the expected net profit of the manufacturer per item.
Solution
Let X be net profit per item. The values of X are 1 (loss) and 5 with corresponding probabilities 0.1 and
0.9. Thus the expected profit E( X) = 1( 0.1) + 5( 0.9) = $4.40
Example 2: A lot consists of 10 motors. It must either be totally rejected or is sold depending on the
outcome of the following procedure. Two motors are chosen at random and inspected .If one or more are
defective, the lot is rejected. Otherwise, it is accepted. Suppose that each motor costs $75 and is sold for
$100. If the lot contains 1 defective motor, what is the manufacturers expected profit?
Solution:
In the lot there are 9 non-defective and 1 defective motors.
Hence P ( onc or morc Jcccti:c) = 1 P ( no Jcccti:c) = 2 . 0
2
10
0
1
2
9
1 =
|
|
.
|
\
|
|
|
.
|
\
|
|
|
.
|
\
|

Thus if X is net profit, Expected net profit is E( X) = 75( 0.2) + 25( 0.8) = $5
Exercise: A coin is tossed three times. Let X be the number of heads. Find the mean of X.
Solution:
The possible outcomes are S = { III, IIE, IEI, EII, EEI, IEI, IEE, EEE}
x 0 1 2 3 Total
P( X = x) 1/8 3/8 3/8 1/8 1
xP( X = x) 0 3/8 6/8 3/8 12/8
Mean of X = E( X) = 8 / 12 ) (
3
0
=
= =
i
i i
x X P x
Definition (Expectation of a Continuous Random Variable): Let X be a continuous random variable with
probability density function f. Then the expected value of X, denoted by E(X), is
defined as:
E( X) = _ xf
X
-
( x) dx

Example: Let X have the P.d.f
< <
=
elsewhere
x x
x f
, 0
1 0 , 2 2
) ( . Find the expected value of X.
Solution:
3
1
3
2
1
3
2
) 2 2 ( ) ( ) (
1
0
3
2
1
0
= =
(
= = =
} }
+

x
x dx x x dx x xf X E

Exercise: Let
< <
=
elsewhere
x x
x f
, 0
1 0 , 2
) ( , find expected value of X
5.6.1.2 Expectation of a function of a random variable
Definition: Let X be a random variable, and g(x) be a function of the random variable X whose domain and
range is the real line.
The expected value of g(x) is defined by
c
=
=
}

continuous is X if x x f x g
discrete is X if x X P x g
x g E
i i
, ) ( ) (
) ( ) (
)) ( (
Example 1: Let V be wind velocity and suppose that P.d.f of V is given as follows
< <
=
elsewhere
v
v f
0
10 0 , 10 / 1
) (

The pressure W on the surface of an air plane wing is given by the relationship W=0.003V
2
. Find E (W).
Example 2: Suppose that X has P.d.f
>
=
otherwise
x
x
x f
, 0
2 ,
8
) (
3

Let W=1/3 X find E (W)
Exercise: Let X have the P.d.f
< <
=
elsewhere
x x
x f
, 0
1 0 , 2 2
) ( . Find E( X
2
) .
5.6.1.3 Properties of expectation
1. If C is any constant value, then E( C) = C
2. E( CX) = C E( X)
3. If X and Y are any two random variables, then E( X + Y) = E( X) + E( Y)
4. If X and Y be two independent random variables, then E( X) = E( X) E( )
5. If a and b are any constant numbers, then E( o + bX) = o + bE( X)
6. E( X) > 0 if X > 0
7. | E( X) | E( | X| )
8. E(g
1
( x) ) E(g
2
( x) ) i f g
1
( X) g
2
( X) for al x
5.6.2 Variance of a random variable and its Properties
Definition: Let X be a random variable .The variance of X, denoted by V(X) is defined as: I( X) =
] ) ) ( ( [
2
X E X E

That is, the mean value of the square of the deviations of X from its mean is called the variance of X or the
variance of the distribution. The positive square root of V(X) is called the standard deviation of X and it is
denoted by s.d(X).
It is worthwhile to observe that
2 2
2 2
2 2 2
)] ( [ ) (
)] ( [ ) ( ) ( 2 ) (
) )] ( [ ) ( 2 ( ) )) ( (( V(X)
X E X E
X E X E X E X E
X E X XE X E X E X E
=
=
= =
or
Ior( X) = ( x
p
X
)
2
P( x
if X is a discrete random variable

Ior( X) = ( x p
X
)
2
X
( x) Jx
-
if X is a continuous random variable
This expression affords an easier way of computing the variance of X.
The standard deviation of X may be interpreted as a measure of the dispersion of the points of the space
relative to the mean value E( X) . We note that if the space contains only one point x for which ( x) > 0,
then s.d=0.
Example1: Let X have the P.d.f
< <
+
=
Otherwise 0
1 x 1 - ,
2
1
) (
x
x f Find the variance of X.
Solution:
In order to find the variance of X we have to first find the expected value of X.
| |
9
2
3
1
3
1
) ( ) E(X V(X) ,
3
1
6 8
)
2 2
(
2
1
) ( ) E(X ,
3
1
)
2 2
(
2
1
) ( ) (
2
2 2
1
1
3 4 2 1
1
3 1
1
2 2 2
1
1
2 1
1
=
|
.
|
\
|
= =
=
(
+ = + =
+
= =
= + ==
+
= =

+

+

} } }
} } }
X E Thus
x x
dx
x x
dx
x
x dx x f x Moreover
dx
x x
dx
x
x dx x xf X E

Example 2: Find the variance of the number of points, which come up on a single toss of a fair die.
Solution:
Let X: The number that come up.
Thus
=
= =
Otherwise 0
6 1,2,3,4,5, x ,
6
1
) ( x X P
5.6.2.1. Properties of Variance
Let X be a random variable:
For any real number k, ) ( ) (
2
X Var k kX Var =
The expression E[ ( X k)
2
] assumes its minimum value when k = E( X)
If Y is another variable that is independent from X, then
Ior( X + ) = Ior( X ) = Ior( X) + Ior( )
If X and Y are not independent, i.e., if they are dependent then
Ior( X + ) = Ior( X) + Ior( ) 2( E( X) E( X) E( ) ) ;
The expression E( X) E( X) E( ) is called the Covariance of X and Y.
Co:( X, ) = E( X) E( X) E( )
Note that: If X onJ are independent then Co:( X, ) = 0

Exercise: A continuous random variable X has P.d.f
>
= =

Otherwise 0
0 x , 2
) (
2x
e
x X P . Find the variance of X.
5.7. Moments and moment generating function
5.7.1. Moments
It is difficult to overemphasize the importance of a moment generating function when it exists. This
importance stems from the fact that the moment generating function of a given random variable is unique
and completely determines the distribution of the random variable; thus if two random variables have the
same moment generating function, they have the same distribution. In fact, it is not every distribution that
has a moment generating function.
Definition (The r
th
Moment about the origin): If X is a random variable, the r
th
moment of the random
variable X, usually denoted by
r
' , is defined as:
p
i
= E( X
) =
P( x
I X is o Jiscrctc ronJom :orioblc

_ x
X
( x) Jx
-
I X is o Continuous ronJom :orioblc

Verbally, a moment is the expectations of the powers of the random variable that has the given distribution.
Moments are defined for a random variable or a distribution.
| |
1 2
2 2
) ( ) ( ) ( ' ' = = X E X E X Var
where E(X) ] E[X =
1
1
= ' and ) E(X =
2
2
'
Definition (Central-Moments): The r
th
moment about the mean of the random variable X, denoted by p
,
is the expected value of ( X p
X
)
; symbolically,
p
= E( X p
X
)
( x
p
X
)
P( x
I X is o Jiscrctc ronJom :orioblc

_ ( x p
X
)
X
( x) Jx
-
I X is o Continuous ronJom :orioblc

Remarks:
The 1st central moment about is zero.
The second central moment is the variance of the random variable.
All odd moments of X about p
X
are 0 if the density function of X is symmetrical about p
X
, provided
that such moment exist.
Example: Let X have the P.d.f
< <
+
=
elsewhere 0
1 x 1 - if
2
1
) (
x
x f
Find a) the first moment about the origin b) the second central moment
Solution:
3
1
12
4
12
1
12
5
4 6 2
1
) ( ) ( )
1
1
2 3 1
1
1
1
1
= = =
(
+ =
|
.
|
\
| +
= = = '

} }
x x
dx
x
x dx x xf X E a

~

( ) ( )
9
2
9
1 3
3
1
3
1
3
1
E(X) have we above a" " part
3
1
24
1
24
7
6 8 2
1
) E(X
) ( ) ( ) )) ( ( ) ( 2 ( )) ( ( )
2
2
1
1
3 4 1
1
2 2
2 2 2 2 2
2
=
= |
.
|
\
|
=
=
=
=
(
+ = |
.
|
\
| +
=
= + = =

}
From
x x
dx
x
x But
X E X E X E X XE X E X E X E b

5.7.2. Moment Generating Functions
Definition (Moment Generating Function): Let X be a random variable with P.d.f f. The expected value
of c
tx
is defined to be the moment generating function of X if in some interval < t < ; > 0. The
moment generating function, denoted by H
X
( t)
, is
H
X
( t)
= E( c
tx
) =
c
tx
i
P( x
) i f X i s a di scr et e r andom var i abl e.

I

_ c
tx
X
( x) Jx i f X i s a cont i nuous r andom var i abl e, and
-

To explain why we refer to this function as moment-generating function, let us substitute for c
tx
its
Taylor series expansion, namely,
c
tx
= 1 +
t
1
x
1
1 !
+
t
2
x
2
2!
+
t
3
x
3
3!
+ +
t
r!
+
For the discrete case, we thus get
H
X
( t)
= c
tx
P( x)
x
= _1 +
t
1
x
1
1!
+
t
2
x
2
2!
+
t
3
x
3
3 !
+ +
t
r!
+ _P( x)
x

= P( x)
x
+ xP( x)
x
+
t
2
2 !
x
2
P( x)
x
+ +
t
r!
x
P( x)
x
+
= 1 + p
1
i
t + p
2
i

t
2
2!
+ + p
i

t
r!
+
and it can be seen that in the Taylor series of moment generating function of X the coefficient of
t
r
!
is p
i
, the
r
th
moment about the origin of the random variable X. In the continuous random variable the argument is the
same.
Notes:
1. From the above expression one can show that;
p
i
=
o
ot
H
X
( t)
|
t=0

i.e., p
i
is the r
th
derivative of H
X
( t)
with respect to t evaluated at t = 0.
2. If o and b are constants, then
I. H
X+u
( t)
= c
ut
E( c
tx
)
II. H
bX
( t)
= E(c
btx
)
III. H
X+u
b
( t)
= c
u
b
t
E _c
t
b
x
]
3. If X and Y are independent r.vs having moments generating functions H
X
( t)
and H
( t)
respectively, then
H
X+
( t)
= H
X
( t)
H
( t)


Proof:
4. In general, If X
1
, X
2
, X
3
, , X
n
are independent random variables with moment generating functions
H
X
i
( t)
; i = 1,2,3, , n then the moment generating function of their sum, H
z
( t)
, where Z = X
1
+ X
2
+ X
3
+ +
X
n
, is given by
H
z
( t)
= H
X
1
( t)
H
X
2
( t)
H
X
n
( t)
= _H
X
i
( t)
n
=1

Proof: (Exercise)
5. Uniqueness theorem: Suppose that X and Y are random variables having moment generating function
H
X
( t)
and H
( t)
respectively. Then X and Y have the same probability distribution iff H
X
( t)
= H
( t)
identically.
Example: Find the moment generating function of the random variable X having P.d.f
< <
=
otherwise 0
2 x 1 - if
3
1
) (x f

Solution:
( )
( )

3
,
1 ) 1 ( ) E(e have we 0 t If
0 t provided
3 3 3
) ( ) (
2
0x
2
2
1
2
1
t
e e
M
Therefore
E
t
e e
t
e
dx
e
dx x f e e E M
t t
t
X
t t tx tx
tx tx t
X
=
= = =
=
=
(
= = = =
} }

Remark: In more advanced courses, we would not work with the moment-generating functions because so
many distributions do not have moment-generating functions.
Example: Consider the random variable X with P.d.f
< <
=
elsewhere 0
1 x 0 if 2
) (
x
x f
Find a) The second moment about origin b) the moment-generating function
2
1
2
) 2 ( ) ( ) ( )
1
0
4 1
0
2 2 2
2
=
(
= = = = '
} }
+

x
dx x x dx x f x X E a
( )
parts) by n integratio .(using ....
) 1 ( 2
) 2 ( ) ( ) ( )
2
1
0
t
e te
dx x e dx x f e e E M b
t t
tx tx tx t
X
+
= = = =
} }
+

Probablity

Transféré par

Informations du document

Description originale:

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Probablity

Transféré par

Droits d'auteur :

Formats disponibles

Pr obabi l i t y and St at i st i cs ( CENG 2007) Addi s Ababa Technol ogy I nst i t ut e

Compi led by: Di baba B./ di boobayu@gmai l.com/ ~ 45 ~

contains those points of the sample space which dont belong to A.

ways, i = 1, 2, . , k, then the

) and is given by the

) for i = 1,2,3, . . . , n, . .. must satisfy the following two conditions:

) 1/4 1/2 1/4

) 1/10 4/10 3/10 2/10

) 1/3 1/2 1/6

} is equivalent to an event, say A, in the range space of X, then

) i = 1,2,3 n. Then the expected

if X is a discrete random variable

I X is o Jiscrctc ronJom :orioblc

I X is o Jiscrctc ronJom :orioblc

) i f X i s a di scr et e r andom var i abl e.

Vous aimerez peut-être aussi