DSME2011 Supplementary Note

- Derivation of Variance formula:

1. Population Variance.

n
! n
!2
2 1X 2 1X
σ = Xi − Xi
n n
i=1 i=1

n
1X
σ2 = (Xi − µ)2 :Definition of variance
n
i=1
n
1 X
Xi2 − 2µXi + µ2


= :Expand bracket
n
i=1
n n n
1 X 1X 1X 2
= Xi2 − 2µXi + µ
n n n
i=1 i=1 i=1
n n
1 X 1 X 1 2
= Xi2 − 2µ Xi + nµ :Take out constants
n n n
i=1 i=1
n n
1 X 1X 2
= Xi2 − 2µ2 + µ2 = Xi − µ2 :Definitin of mean
n n
i=1 i=1
n
! n
!2
1X 2 1X
= Xi − Xi :Definitin of mean
n n
i=1 i=1

2. Variance of a Random Variable. Var[X] = E[X 2 ] − (E[X])2

h i
Var[X] = E (X − E[X])2 :Definition of variance
h i
= E X 2 − 2E[X]X + (E[X])2 :Expand bracket
h i
= E X 2 − E [2E[X]X] + E (E[X])2
 

= E X 2 − 2E[X]E [X] + (E[X])2 E [1]

 
:Take out constants

= E X 2 − 2 (E[X])2 + (E[X])2
 

= E X 2 − (E[X])2
 

1
- Derivation of Covariance formula of random variables X, Y :

Cov[X, Y ] = E[XY ] − E[X]E[Y ]

Cov[X] = E [(X − E[X]) (Y − E[Y ])] :Definition of Covariance

= E [XY − E[X]Y − E[Y ]X + E[X]E[Y ]] :Expand bracket

= E [XY ] − E [E[X]Y ] − E [E[Y ]X] + E [E[X]E[Y ]] :Separate each terms

= E [XY ] − E [X] E [Y ] − E [Y ] E [X] + E[X]E[Y ] :Take out constants

= E[XY ] − E[X]E[Y ]

- Derivation of Variance-of-Sum Formula:

For any constants a, b and random variables X, Y ,

Var[aX + bY ] = a2 Var[X] + b2 Var[Y ] + 2ab · Cov[X, Y ]

= a2 Var[X] + b2 Var[Y ] + 2ab · σx σy · ρxy

h i
Var[aX + bY ] = E (aX + bY − E[aX + bY ])2 :Definition of Variance
h i
= E (aX − E[aX] + bY − E[bY ])2 :Property of Expectation
h i
= E {(aX − E[aX]) + (bY − E[bY ])}2 :Group terms
h i
= E {(aX − aE[X]) + (bY − bE[Y ])}2 :Take out constants

= E (aX − aE[X])2 + 2(aX − aE[X])(bY − bE[Y ]) + (bY − bE[Y ])2

 
:Expand

= E (aX − aE[X])2 + E [2(aX − aE[X])(bY − bE[Y ])] + E (bY − bE[Y ])2

   

= E a2 (X − E[X])2 + E [2ab(X − E[X])(Y − E[Y ])] + E b2 (Y − E[Y ])2

   

= a2 E (X − E[X])2 + 2abE [(X − E[X])(Y − E[Y ])] + b2 E (Y − E[Y ])2

   

= a2 Var[X] + 2abCov[X, Y ] + b2 Var[Y ] :Definition of Variance and Covariance