Probability and Statistics

B Madhav Reddy

madhav.b@srmap.edu.in

B Madhav Reddy Probability and Statistics 1 / 15

Recall..

Discrete Continuous
Conditional probability Conditional probability
mass function density function
pX ∣Y (x∣b) = p(x ,b)
pY (b) fX ∣Y (x∣b) = ffY(x(b)
,b)

P{X ∈ A∣Y = b} = ∑ pX ∣Y (x∣b) P{X ∈ A∣Y = b} = ∫A fX ∣Y (x∣b) dx

x ∈A
a
FX ∣Y (x∣b) = ∑ pX ∣Y (x∣b) FX ∣Y (x∣b) = ∫−∞ fX ∣Y (x∣b) dx
a≤x

☀ If X and Y are independent,

● Discrete case: pX ∣Y (x∣b) = pX (x)
● Continuous case: fX ∣Y (x∣b) = fX (x)
Ô⇒ If X and Y are independent, it is enough to work with marginal
functions instead of conditional functions.

B Madhav Reddy Probability and Statistics 2 / 15

Recall..

Definition

⎧
⎪ ∑ ∑ h(x)g(y )p(x, y ), if X and Y are discrete
⎪
⎪
⎪
⎪
⎪
⎪
⎪
x y
⎪
E [h(X )g(Y )] = ⎨
⎪
⎪
⎪
⎪ ∫−∞ ∫−∞ h(x)g(y )f (x, y ) dx dy , if X and Y
∞ ∞
⎪
⎪
⎪
⎪
⎪
⎩ are continuous

Fact 12. If X and Y are independent, then, for any functions h and g,

E [h(X )g(Y )] = E [h(X )] E [g(Y )]

In particular, for independent random variables X and Y ,

E [XY ] = E [X ] E [Y ]

B Madhav Reddy Probability and Statistics 3 / 15

Recall..

Definition
The covariance between any two random variables X and Y, denoted by
Cov (X , Y ), is defined by

Cov (X , Y ) = E [(X − E [X ]) (Y − E [Y ])]

Fact 13. Cov (X , Y ) = E [XY ] − E [X ] E [Y ]

☀ If X and Y are independent, then by Fact 12, Cov (X , Y ) = 0

B Madhav Reddy Probability and Statistics 4 / 15

Fact 14.
1 Cov (X , Y ) = Cov (Y , X )
2 Cov (X , X ) = Var (X )
3 Cov (aX , Y ) = aCov (X , Y )

Cov ( ∑ Xi , ∑ Yj ) = ∑ ∑ Cov (Xi , Yj )

n m n m
4
i=1 j=1 i=1 j=1
n n n
Now, Var (∑ Xi ) = Cov (∑ Xi , ∑ Xi )
i=1 i=1 i=1
n m
= ∑ ∑ Cov (Xi , Xj )
i=1 j=1

= ∑ ∑Cov (Xi , Xj ) + ∑ ∑Cov (Xi , Xj )

i=j i≠j
n
= ∑ Var (Xi ) + 2∑ ∑Cov (Xi , Xj )
i=1 i<j

B Madhav Reddy Probability and Statistics 5 / 15

 Fact 15. Var ( ∑ Xi ) = ∑ Var (Xi ) + 2∑ ∑Cov (Xi , Xj )
n n

i=1 i=1 i<j

● If X1 , X2 , . . . Xn are paiwise independent, then

n n
Var (∑ Xi ) = ∑ Var (Xi )
i=1 i=1

Definition
The correlation coeffiecient of two random variables X and Y , denoted
by ρ(X , Y ), is defined, as long as Var (X ) and Var (Y ) are positive, by

Cov (X , Y )
ρ(X , Y ) = √ √
Var (X ) Var (Y )

Fact 16. −1 ≤ ρ(X , Y ) ≤ 1, for any two random variables X and Y

B Madhav Reddy Probability and Statistics 6 / 15

Exercise: The random variables X and Y have a joint density function
given by
⎧
⎪ 1 e −(y + x /y ) , x > 0, y > 0
⎪
f (x, y ) = ⎨ y
⎪
⎪
⎩ 0, otherwise
Find the correlation coefficient, ρ (X , Y ), of X and Y .
Procedure:
Step-1: Calculate the marginal densities
∞ ∞
fX (x) = ∫ f (x, y ) dy and fY (y ) = ∫ f (x, y ) dx
−∞ −∞

Step-2: Calculate the expectations

E [X ] = ∫−∞ xfX (x) dx E [Y ] = ∫−∞ yfY (y ) dy

∞ ∞

E [X 2 ] = ∫−∞ x 2 fX (x) dx E [Y 2 ] = ∫−∞ y 2 fY (y ) dy

∞ ∞

B Madhav Reddy Probability and Statistics 7 / 15

 Step-3: Calculate the joint expectation
∞ ∞
E [XY ] = ∫ ∫ xyf (x, y ) dx dy
−∞ −∞

Step-4: Calculate Cov (X , Y ), Var (X ) and Var (Y )

Cov (X , Y ) = E [XY ] − E [X ] E [Y ]

Var (X ) = E [X 2 ] − (E [X ])2 and Var (Y ) = E [Y 2 ] − (E [Y ])2

Step-5: Finally,

Cov (X , Y )
ρ(X , Y ) = √ √
Var (X ) Var (Y )

☀ −1 ≤ ρ(X , Y ) ≤ 1

B Madhav Reddy Probability and Statistics 8 / 15

 The correlation coefficient is a measure of the degree of linearity
between X and Y
A positive value of ρ(X , Y ) indicates that Y tends to increase when
X does, whereas a negative value indicates that Y tends to decrease
when X increases
If ρ(X , Y ) = 0, then X and Y are said to be uncorrelated

Definition
Random variables X1 , X2 , . . . are said to be identically distributed if each
of Xi has the exactly same distribution. That is,

FXi (x) = FXj (x), for every x

B Madhav Reddy Probability and Statistics 9 / 15

Theorem (Central limit theorem)
Let X1 , X2 , . . . be a sequence of independent and identically distributed
random variables, each having mean µ and variance σ 2 . Then the
distribution of
X1 + X2 + ⋯ + Xn − nµ
√
σ n
tends to the standard normal as n → ∞.

−nµ
That is, if Yn = X1 + X2 + ⋯ + Xn and Zn = Yn√
σ n
, then

Yn − nµ
FZn (a) = P { √ ≤ a} Ð→ Φ(a) = √ ∫ e − x /2 dx
1 a 2

σ n 2π −∞
as n → ∞

Eventually, everything will be normal!

B Madhav Reddy Probability and Statistics 10 / 15

This theorem is very powerful because it can be applied to random
variables X1 , X2 , . . . having virtually any thinkable distribution with
finite expectation and variance.
As long as n is large (the rule of thumb is n > 30), one can use
Normal distribution to compute probabilities about the sum Yn

B Madhav Reddy Probability and Statistics 11 / 15

Example: A disk has free space of 330 megabytes. Is it likely to be
sufficient for 300 independent images, if each image has expected size of 1
megabyte with a standard deviation of 0.5 megabytes?
Solution: Let Xi denote the size of i th image in megabytes
Given that X1 , X2 , . . . are identically distributed with mean 1 and standard
deviation 0.5
If we denote Yn = X1 + X2 + ⋯ + Xn , then we need to find P {Yn ≤ 330} for
n = 300
Since n is large (300 in our case), we use the central limit theorem

B Madhav Reddy Probability and Statistics 12 / 15

 Y300 − 300(1) 330 − 300(1)
P {Y300 ≤ 330} = P { √ ≤ √ }
(0.5) 300 (0.5) 300
= P {Z300 ≤ 3.46}
≈ Φ(3.46) (by central limit theorem)
≈ 0.9997
This probability is very high, hence, the available disk space is very likely
to be sufficient.

B Madhav Reddy Probability and Statistics 13 / 15

Example: You wait for an elevator, whose capacity is 2000 lbs. The
elevator comes with ten adult passengers. Suppose your own weight is 150
lbs, and you heard that human weights are normally distributed with the
mean of 165 lbs and the standard deviation of 20 lbs. Would you board
this elevator or wait for the next one? In other words, is overload likely?
Solution: Let Xi denote the weight of i th person in the lift and
Yn = X1 + X2 + ⋯ + Xn
Overload happens if Y10 + 150 > 2000. Hence we need to find
P {Y10 + 150 > 2000}

B Madhav Reddy Probability and Statistics 14 / 15

 P {Y10 + 150 > 2000} = P {Y10 > 1850}
Y10 − 165(10) 1850 − 165(10)
=P{ √ > √ }
20( 10) 20( 10)
= P {Z10 > 3.16}
= 1 − P {Z10 ≤ 3.16}
≈ 1 − Φ(3.16) (by central limit theorem)
≈ 1 − 0.9992 = 0.0008

Thus, there is only 0.08% chance of overload. Hence it is safe to take the
elevator!

B Madhav Reddy Probability and Statistics 15 / 15