CENTRAL LIMIT THEOREMS

Consider the sequence
  and take  = 

 . Consider also
where
  and
  are constants of particular interest:
 


  
1. Under what conditions can we find
  and
  such that 


2. If
  and
  exists, what are the possible distributions of z ?

(non-degenerate) - standard normal variate.

  


3. What sufficient conditions will allow us to determine that
specified distributions?

Classical Central Limit Theorem (CLT)

Let  = [ ],  = [ ], ~(0,1)

  ~(0,1)
 #[ ]
$%[ ]
If

We say
  obeys the CLT.

 '
Note: 
     iff &  &
( 

     
    
'
* (+)  *(+)  &  &
.-. ( 

  ./ 
   
   

Central Limit Theorem 1

CLT for independent, identically distributed (iid) random variables

Let
  be iid with [0

] < and [0
] < . Then the sequence
obeys the CLT.

Proof: WLOG, let  [0

] = 0 and [0
] = 1.
Let  = 
 . Then  [ ] = 0, [ ] = 3
For any 4 , show that as 3 
D
 1 @
78 4; = < ? BA
A C
93 92>
E

FA
i.e * (+)  ?

A
9
Or characteristic function of characteristic function
of (0,1)
 

G (+) = * HI (+) = J * HI (+) = J * H@ (+)

9 9 9



 
= 8* (+); = K*D@ L NO
M
HI
9
9

Expanding in terms of Taylor Series:

F A
L N R
= P1 +  + 0 SL N TU
9 M
R 9

Central Limit Theorem 2

Recall: Let
V  be complex. If V  V then
V 
lim L1 + N = ? '
E 3
V 1

lim Z1 + + 0 S T[ = ? '
E 3 3

FA
Let V = . Then limE * (+) = ?
MA
A
R
@
Note: limE (1 +  ) = ?


@
limE (1 + 4 ) = ? D
H

Central Limit Theorem 3

Conditions under which a CLT holds
 #[ ]
$%[ ]
1. The quality must be the sum of small parts in the sense that
none of the terms in the sum dominates e.g.
 iid,  [0
] = 0 and
[0
] = 1 then = +. . . +
 ]@ ]
9 9 9

and 7 Kmax
`` a I a cO = 7 Kd
ea I afO 
7 Ka I a cO
] ] ]
9 9 9

= 37 Ka @ a cO = 3 gKaH@ aijO 1 C& ( ), where F is the

]
9 9
common CDF of the Xs.

3 gKa H aijO C&()

]A
jA
9

= 0 as 3 

j A gmnHno9pq ] A kl(])

since this is tail end probabilities.

 

2. The sequence must not behave irregularly compared to
(0,1) i.e.  [] = 0 and [] = 1 e.g.
  independent

7[0r r ] =

Rr A

7[0r t ] = 1

rA

 [0 ] = 0 and  ] = 1

 #[ ]
=

$%[ ] 9

If CLT is to hold, then limE 7 K = 0O = 0


9

But 7 K = 0O 7[r = 0,v = 1, 2, . . . , 3]


9

= wr
7[r = 0] = wr
L1 N  .xx/ V > 0

rA E
Central Limit Theorem 4
Linderbergs CLT

Let
  independent with finite variances. Let

 [ ] = z , [{R ],  = 
 ,  [ ] = |R , [ ] = | .

r
g
n] } ( zrR )C&r ()  0 as 3 

| ~ n| 
If A

 #[ ] 
  (0,1)
  obeys the CLT.
|
Then

Example:

0r ~(v, v),
0r  is independent  [0r ] = 0 [0r ] =

rA

 
vR 1 1 3(3 + 1)(23 + 1) 1
| =
R
= vR = 3 |
3 3 3 6 9
r
r

3A
=
3
The Lindebergs condition is
 
9 9 R
<  C&] () =
R
< C
3 3 A 2v


A
n]n
r

[ r,r]n]n r

 r
g C  0 as 3  since the area
]A


A Rr
[ ,]n]n

of integration  a tail event.

Central Limit Theorem 5


R
Given > 0, take 3 > L N . For this 3 , > 3
A

0r , 0r ~(v, v) obeys CLT.

Since Linderbergs condition is satisfied, then for independent

Liapunovs Condition

Let
  be independent sequence. If for some  > 0:

i.  [n0r nR ] < v
limE A   [n0r zr nR ] = 0

|
ii.

Then Lindebergs.

Proof: Show Lindebergs condition is satisfied when Liapunovs

condition is satisfied.
WOLOG, we can assume  [0r ] = zr = 0v.

r
gnn  R C&r = r
gnn| (| )  R C&r ()

|
A A
| 
Show:

r
gnn| n nR C&r ()

A
| 

r
g n nR C&r ()

A
|

Take limit as 3  ,
 0  0

Central Limit Theorem 6

Fellers Condition

Fellers Theorem. Let
0  independent, m0 R q < 3.

 #[ ]
 ~
(0,1)
$%[ ]
i.
ii. E max
`r` 7 [n0r zr n | ] = 0
 > 0, limE
Then Lindebergs condition holds.
Note: If Lindebergs condition does not hold AND Fellers condition
holds, then CLT does not hold.

Example 1:
0  are iid with finite variance, say, { R .

i. CLT holds by CLT of iid random variable.
ii. Fellers condition
lim max 7[n0r zr n | ] = lim max 7 Kn0r zr n 3{ R O
E
`r` E
`r`
= lim 7mn0r zr n 93{ R q =0
E
by Chebyshevs inequality

7[n0n ]
#m A q
A

Then Lindebergs condition holds.

To show directly:

r
g[n ( zr )R C&r ()  0 as 3  .

| ~ }~ ni| ]
A

Central Limit Theorem 7

Example 2. Let
0r  independent sequence of random variables

7[0r = v R ] = , 7[0r = v R ] = , 7[0r = 0] =

Rr A
R r A

 [0r ] = 0,  [0rR ] = , |R = [ ]

rA 

Then, Lindebergs condition does not hold (but does not imply that CLT
does not hold).
Does Fellers condition hold?

limE max
`r` 7 n0r n = lim 7 n0r n since
A A
E

maximum is at 0 .

lim 2 = = 0 CLT does not hold.

E A
RA

R

Berry-Essen Theorem

Let
0  be independent random variables with finite mean and finite
variance. Let & be the CDF of
 #[ ]
|

Let standard normal CDF

sup
n& () ()n r
 [n0r zr n ] for some V <
'
|
Then,
and n.

Central Limit Theorem 8

CLT for pdfs:

If
0  are absolutely continuous iid random variables with finite mean
and finite variance, there exists a k for which the pdf f satisfies:
n()n v
Then lim supn () (s)n = 0
E

where () is the pdf of , and () the pdf of standard normal.

 #[ ]
|

CLT:

Let 
, R , . ..iid with  [ ] = z and ( ) = { R > 0
} k } ] @
   (0,1) as 3  i.e 7    g E 9R ? A C

A
 
9 9

Proof: By the continuity theorem for characteristic functions, it suffices

to show that
A
 }
 ( )  ?
A where  =

9
I
But  ( ) =  ?
[(]@ })...(] })]
9

= K(]I }) L NO

9

Central Limit Theorem 9

 R A 
= 81 + L N + 0 L N;


9 R 
by the moment expansion of

the characteristic function


= K1 + 0 L NO
A

R 
 A
= K1 O  ? A as 3 
A
R

Applications of CLT
1. Generating random variables which are approximately (0,1)- a
quick and dirty method is to generate k(0,1) and take the
average.
2. Approximations of distributions by normal distribution

If 0~ (3, ), we know that 0 has same pdf as

a. Normal approximations to binomial

+ R +. . . + where the  are independent with
7( = 1) = and 7( = 0) = 1
  [  ] = 3 and [  ] = 3(1 )

k
 (0,1).
 
(
)
By CLT,

Hence, for large 3, 0 ~(3, 3(1 ))

Thus, for large 3:
7[0  ] 7[  ] where  ~3, 3(1 ).

7[0  ] 7 K  + O for small n (continuity correction)

Central Limit Theorem 10

Note: The latter approximation is same as approximating 7[0  ] =
7 K0  0 + O. Roughly speaking, the approximation is good

R R
if the mean, np, is more or less on the middle far from 0 or n, i.e. if
3 > 43(1 ) and 3 < 43(1 ).

b. Approximation of gamma distribution

If 0 ~(3, ) then 0 has the same distribution as the

sum of n independent exponential .

[Recall:  (0) = 3 and (0) = 3 R ]

Hence,  ~(0,1) i.e. 0 ~(3, 3 R ) for n (integer) large.

 
9

Central Limit Theorem 11