Statistics 200

Winter 2009
Homework 5 Solutions

Problem 1 (8.16)
X1 , . . . , Xn i.i.d. with density function f (x|) =

1
2



exp |x|

(a) (c) (See HW 4 Solutions)

(d) According to Corollary A on page 309 of the text, the maximum likelihood estimate is a function of a
sufficient statistic T . In part (b), the maximum likelihood estimate was found to be
n

M LE =

1X
|xi |
n i=1

Therefore, a sufficient statistic T (X1 , X2 , . . . , Xn ) is given by:

T (X1 , X2 , . . . , Xn ) =

n
X

|Xi |

i=1

Problem 2 (8.52)
X1 , . . . , Xn i.i.d. with density function f (x|) = ( + 1) x ,

,0 x 1

(a)
1

Z
E[X] =

xf (x|)dx
0
1

x(+1) ( + 1)dx

=
0

1
+ 1 (+2) 
x

+2
0
+1
=
+2
=

Therefore, a method of moments estimate of is given by:

where
1 =

1
n

Pn

i=1

M M + 1

1 =

+2
 MM 

1 M M + 2 = M M + 1
(
1 1) M M = 1 2
1
1

1
M M =

1 1

n.
Xi = X

(b)
l() =

n
X

[log( + 1) + log(xi )]

i=1
n

X
d
n
l() =
+
log(xi )
d
+ 1 i=1
0=

n
M LE + 1

n
X

log(xi )

i=1

n
1
M LE = Pn
i=1 log(xi )

(c)

2
log
f
(X|)
2
 2


= E
(log(
+
1)
+

log
X)
2
 


1
= E
+ log X
+ 1


1
= E
( + 1)2
1
=
( + 1)2


I() = E

1
nI()
( + 1)2
=
n

h
i
Var M LE

(d) According to Corollary A on page 309 of the text, the maximum likelihood estimate is a function of a
sufficient statistic T . In part (b), the maximum likelihood estimate was found to be
n
M LE = Pn
1
i=1 log(xi )
2

Therefore, a sufficient statistic T (X1 , X2 , . . . , Xn ) is given by:

T (X1 , X2 , . . . , Xn ) =

n
X

log(Xi )

i=1

Problem 3 (8.59)
Let I denote the event that a pair of twins is identical, so P (I) = .
(a)
P (M M ) = P (M M |I)P (I) + P (M M |I C )P (I C )
  
1
1
1
(1 )
= +
2
2
2
1+
=
4
P (F F ) = P (F F |I)P (I) + P (F F |I C )P (I C )
  
1
1
1
= +
(1 )
2
2
2
1+
=
4
P (M F ) = 1 (P (M M ) + P (F F ))
1+
=1
2
1
=
2
(b) We will assume that n sets of twins are sampled, so n1 + n2 + n3 = n.

n

n

n
1+ 1
1+ 2
1 3
lik() =
+
+
4
4
2




1+
1

l() = (n1 + n2 ) log

+ n3 log
4
2
n1 + n2
n3
d
l() =

d
1+
1
n1 + n2
n3

0=

1+
M LE
1
M LE
n1 + n2 n3

M LE =
n
Now to compute the variance of
M LE , we will rewrite
M LE as
n1 + n2 n3
n
n1 + n2 (n n1 n2 )
=
n
2(n1 + n2 ) n
=
n

M LE =

Then the variance of the MLE can be computed as



2(n1 + n2 ) n
Var[
M LE ] = Var
n
4
= 2 Var[n1 + n2 ]
n
4
= 2 (Var[n1 ] + Var[n2 ] + 2Cov(n1 , n2 ))
n

We note that n1 and n2 are both Binomial random variables with n trials and success probability
so



3
1+
Var[n1 ] = Var[n2 ] = n
4
4

1+
4 ,

th
th
Now
Pn we define Yi =
Pn1{i set of twins is M M } and Xi = 1{i set of twins is F F }. Clearly n1 =
i=1 Yi and n2 =
i=1 Xi , and also Yi Xi = 0 since a given set of twins cannot be both two males
and two females. Using these definitions, we have

Cov(n1 , n2 ) = E[n1 n2 ] E[n1 ]E[n2 ]

! n
n
X
X
n(1 + ) n(1 + )
= E
Yi
Xj
4
4
i=1
j=1


2
n
X
X
1+
2

=E
Yi Xi +
Yi Xj n
4
i=1
i6=j

" n #

2
X
X
1+
2

=E
0 +E
Yi Xj n
4
i=1
i6=j

1+
= 0 + (n n)E[Yi ]E[Xj ] n
4

2

2
1+
1+
= (n2 n)
n2
4
4
2

1+
= n
4
2

2

Substituting these results back into the expression for Var[

M LE ], we have
" 






2 !#
4
1+
3
1+
3
1+
Var[
M LE ] = 2 n
+n
+ 2 n
n
4
4
4
4
4


2
1 (1 + )(3 ) (1 + )
=
+
n
2
2
(1 + )4
=
2n
2(1 + )
=
n

Problem 4 (8.68)
1 x
e
X1 , . . . , Xn i.i.d. with probability mass function function p(x|) = x!
Pn
(a) To show that T = i=1 Xi is sufficient for , we first note that T has a Poisson distribution with
parameter n, so we have:

P (X1 = x1 , X2 = x2 , . . . , Xn = xn | T = t)
=

P (X1 = x1 , X2 = x2 , . . . , Xn = xn , T = t)
P (T = t)

Pn1 
P X1 = x1 , X2 = x2 , . . . , Xn = t i=1 xi
P (T = t)
hQ

n1
i=1

ih

Pn1
Pn1  i
xi e /xi ! (t i=1 xi ) e / t i=1 xi !
(n)t en /t!

e(n1)

Pn

i=1

xi

Q



Pn1
Pn1 
1/xi ! (t i=1 xi ) e / t i=1 xi !
(n)t en /t!

en t

Q

n1
i=1

h 
Pn1  i
1/xi ! 1/ t i=1 xi !
(n)t en /t!

Q
=

n1
i=1

n1
i=1

h 
Pn1  i
1/xi ! 1/ t i=1 xi !
nt /t!

Since the distribution of X1 , . . . , Xn given T does not depend on , T =

Pn

i=1

Xi is sufficient.

(b) To show that X1 is not sufficient, we again compute the distribution of X1 , . . . , Xn given X1 :
P (X1 = x1 , X2 = x2 , . . . , Xn = xn | X1 = x1 )
=

P (X1 = x1 , X2 = x2 , . . . , Xn = xn , X1 = x1 )
P (X1 = x1 )

P (X1 = x1 , X2 = x2 , . . . , Xn = xn )
P (X1 = x1 )
Qn

xi
/xi !
i=1 e
t

e /x1 !

n
Y

xi e /xi !

i=2

Since this distribution still depends on , X1 is not sufficient.

5

(c) According to Theorem A of Section 8.8.1, the statistic T is sufficient if and only if the density
f (x1 , . . . , xn |) can be factored as
f (x1 , . . . , xn |) = g(T (x1 , . . . , xn ), )h(x1 , . . . , xn )
Pn
For the Poisson density and the statistic T = i=1 Xi , we can write
f (x1 , . . . , xn |) =

n
Y
xi e

xi !

i=1

#
" n
i Y
1
=
e
x!
i=1 i
" n
#

 Y 1
= T en
x!
i=1 i
h

Pn

i=1 xi n

= g(T (x1 , . . . , xn ), )h(x1 , . . . , xn )

where
g(T, ) = T en
n
Y
1
h(x1 , . . . , xn ) =
x
!
i=1 i

Problem 5
X1 , . . . , Xn i.i.d. with density function f (x|, 2 , p) = pf1 (x|) + (1 p)f2 (x|, 2 ), where


1
(x )2
f1 (x|) = exp
2
2
is the N (, 1) density, and


(x )2
f2 (x|, ) =
exp
2 2
2 2
1

is the N (, 2 ) density. Then the expectation of a random variable with this mixture density is given by:
Z
E[Xi ] =
xf (x|, 2 , p)dx

Z


=
x pf1 (x|) + (1 p)f2 (x|, 2 ) dx

Z
Z
=p
xf1 (x|)dx + (1 p)
xf2 (x|, 2 )dx

= p + (1 p)
=

To calculate the variance of a random variable with this mixture density, we use the fact that Var[Xi ] =
E[Xi2 ] (E[Xi ])2 , where E[Xi2 ] is given by:
Z
E[Xi2 ] =
x2 f (x|, 2 , p)dx

Z


x2 pf1 (x|) + (1 p)f2 (x|, 2 ) dx
=

Z
Z
2
=p
x f1 (x|)dx + (1 p)
x2 f2 (x|, 2 )dx

= p(1 + ) + (1 p)( + )
= 2 + p + (1 p) 2
So we have
Var[Xi ] = E[Xi2 ] (E[Xi ])2
= 2 + p + (1 p) 2 2
= p + (1 p) 2

Problem 6
For the mixture density of problem 5, the variance of the sample mean is given by
Var[Xi ]
n
p + (1 p) 2
=
n

n] =
Var[X

And for large n, we have that


n (Mn M ) d N

1
0, 2
4f (M )

where M is the true median of the distribution. Since this distribution is symmetric about , we have that
M = , and therefore
f 2 (M ) = f 2 ()

2
1
1
= p + (1 p)
2
2 2
1
1
1
= p2
+ p(1 p)2
+ (1 p)2
2
2
2 2

Thus we have
p
Var[ (n) (Mn M )]

1
4f 2 (M )

1
4f 2 (M )
1
nVar[Mn ] 2
4f (M )
1
Var[Mn ]
4nf 2 (M )
p
Var[ (n)Mn ]

Var[Mn ]

1


1
1
1
+ p(1 p)2 2
+ (1 p)2 2
4n p2 2
2

When p = 0.9 and = 5,

0.9 + 0.1(25)
n
1
= 3.4
n

n] =
Var[X

Var[Mn ]

1
0.92 2

4n
1
= 1.8559
n

1


1
1
+ 0.9(0.1)2 10
+ 0.12 50

So the ratio of the asymptotic variances for p = 0.9 and = 5 is

n]
Var[X
3.4/n
=
Var[Mn ]
1.8559/n
= 1.8320
n is given by
A confidence interval for based on X
n z1/2
X

q
n]
Var[X

In order for a 95% confidence interval to have length 0.1, we must have
q
n ] = 0.05
z10.05/2 Var[X
r
3.4

1.96
= 0.05
n
r
n

39.2 =
3.4
n

1536.64 =
3.4

n = 5224.576
n with length 0.1.
Thus a sample of size 5225 is needed to give a 95% confidence interval based on X
Similarly, a confidence interval for based on Mn is given by
p
Mn z1/2 Var[Mn ]
8

In order for this 95% confidence interval to have length 0.1, we must have
p
z10.05/2 Var[Mn ] = 0.05
r
1.8559

1.96
= 0.05
n
r
n

39.2 =
1.8559
n

1536.64 =
1.8559

n = 2851.85
So a sample of size 2852 is needed to give a 95% confidence interval based on Mn with length 0.1.

Problem 7
X1 , . . . , Xn i.i.d. according to the Cauchy distribution with density function
f (x|) =

1
1

1 + (x )2

If n is the sample median, we have for large n that




n n M d N 0,

1
2
4f (M )

where M is the true median of the distribution. Since this distribution is symmetric about , we have that

2
M = , and therefore f 2 (M ) = f 2 () = 1 . This gives





2
n n d N 0,
4


n n
d N (0, 1)
/2

Using this limiting distribution, we have




 1
1
1



= P n
P  n 
5
5
5

 n 
n 
=P
n n
5
5


n n
n
2
n
2

= P
5
/2
5
 
 
2 n
2 n
=

5
5
!
!

2 101
2 101
=

5
5
= 0.7993
9

If instead we use an efficient estimator n that satisfies





1

n n d N 0,
I()


n n d N (0, 2)


n n

d N (0, 1)
2

then we have
P




 1
1
1


= P n
 n 
5
5
5


 n 

n
n n
=P
5
5



n n
n
n
1
1

= P

5
5
2
2
2
 
 
n
n

=
5 2
5 2
!
!

101
101

=

5 2
5 2
= 0.8448

Thus the efficient estimator n has a higher probability of being within 0.2 of the true value , as expected.

10