Académique Documents
Professionnel Documents
Culture Documents
,
R
k
, and E|X
1
|
k
< .
Let
j
= EX
j
1
be the jth moment of P and let
j
=
1
n
n
i=1
X
j
i
be the jth sample moment, which is an unbiased estimator of
j
, j = 1, ..., k.
Typically,
j
= h
j
(), j = 1, ..., k, (1)
for some functions h
j
on R
k
.
By substituting
j
s on the left-hand side of (1) by the sample moments
j
, we obtain a
moment estimator
, i.e.,
satises
j
= h
j
(
), j = 1, ..., k,
which is a sample analogue of (1).
This method of deriving estimators is called the method of moments.
An important statistical principle, the substitution principle, is applied in this method.
Let = (
1
, ...,
k
) and h = (h
1
, ..., h
k
).
Then = h(
).
If the inverse function h
1
exists, then the unique moment estimator of is
= h
1
( ).
When h
1
does not exist (i.e., h is not one-to-one), any solution of = h(
) is a moment
estimator of ;
if possible, we always choose a solution
in the parameter space .
In some cases, however, a moment estimator does not exist (see Exercise 111).
Assume that
= g( ) for a function g.
If h
1
exists, then g = h
1
.
If g is continuous at = (
1
, ...,
k
), then
is strongly consistent for , since
j
a.s.
j
by
the SLLN.
If g is dierentiable at and E|X
1
|
2k
< , then
is asymptotically normal, by the CLT
and Theorem 1.12, and
amse
() = n
1
[g()]
g(),
where V
j
.
Furthermore, the n
1
order asymptotic bias of
is
(2n)
1
tr
_
2
g()V
_
.
1
Example 3.24. Let X
1
, ..., X
n
be i.i.d. from a population P
=
_
X,
1
n
n
i=1
(X
i
X)
2
_
=
_
X,
n 1
n
S
2
_
.
Note that
X is unbiased, but
n1
n
S
2
is not.
If X
i
is normal, then
is sucient and is nearly the same as an optimal estimator such as
the UMVUE.
On the other hand, if X
i
is from a double exponential or logistic distribution, then
is not
sucient and can often be improved.
Consider now the estimation of
2
when we know that = 0.
Obviously we cannot use the equation
1
= to solve the problem.
Using
2
=
2
=
2
, we obtain the moment estimator
2
=
2
= n
1
n
i=1
X
2
i
.
This is still a good estimator when X
i
is normal, but is not a function of sucient statistic
when X
i
is from a double exponential distribution.
For the double exponential case one can argue that we should rst make a transformation
Y
i
= |X
i
| and then obtain the moment estimator based on the transformed data.
The moment estimator of
2
based on the transformed data is
Y
2
= (n
1
n
i=1
|X
i
|)
2
, which
is sucient for
2
.
Note that this estimator can also be obtained based on absolute moment equations.
Example 3.25. Let X
1
, ..., X
n
be i.i.d. from the uniform distribution on (
1
,
2
), <
1
<
2
< .
Note that
EX
1
= (
1
+
2
)/2
and
EX
2
1
= (
2
1
+
2
2
+
1
2
)/3.
Setting
1
= EX
1
and
2
= EX
2
1
and substituting
1
in the second equation by 2
1
2
(the rst equation), we obtain that
(2
1
2
)
2
+
2
2
+ (2
1
2
)
2
= 3
2
,
which is the same as
(
2
1
)
2
= 3(
2
2
1
).
Since
2
> EX
1
, we obtain that
2
=
1
+
_
3(
2
2
1
) =
X +
_
3(n1)
n
S
2
2
and
1
=
1
_
3(
2
2
1
) =
X
_
3(n1)
n
S
2
.
These estimators are not functions of the sucient and complete statistic (X
(1)
, X
(n)
).
Example 3.26. Let X
1
, ..., X
n
be i.i.d. from the binomial distribution Bi(p, k) with unknown
parameters k {1, 2, ...} and p (0, 1).
Since
EX
1
= kp
and
EX
2
1
= kp(1 p) + k
2
p
2
,
we obtain the moment estimators
p = (
1
+
2
1
2
)/
1
= 1
n1
n
S
2
/
X
and
k =
2
1
/(
1
+
2
1
2
) =
X/(1
n1
n
S
2
/
X).
The estimator p is in the range of (0, 1).
But
k may not be an integer.
It can be improved by an estimator that is
k rounded to the nearest positive integer.
Example 3.27. Suppose that X
1
, ..., X
n
are i.i.d. from the Pareto distribution Pa(a, ) with
unknown a > 0 and > 2 (Table 1.2, page 20).
Note that
EX
1
= a/( 1)
and
EX
2
1
= a
2
/( 2).
From the moment equation,
(1)
2
(2)
=
2
/
2
1
.
Note that
(1)
2
(2)
1 =
1
(2)
.
Hence
( 2) =
2
1
/(
2
2
1
).
Since > 2, there is a unique solution in the parameter space:
= 1 +
_
2
/(
2
2
1
) = 1 +
_
1 +
n
n1
X
2
/S
2
and
a =
1
(
1)
=
X
_
1 +
n
n1
X
2
/S
2
__
1 +
_
1 +
n
n1
X
2
/S
2
_
.
3
Exercise 108. Let X
1
, ..., X
n
be a random sample from the following discrete distribution:
P(X
1
= 1) =
2(1 )
2
, P(X
1
= 2) =
2
,
where (0, 1) is unknown.
Note that
EX
1
=
2(1 )
2
+
2
2
=
2
2
.
Hence, a moment estimator of is
= 2(1
X
1
), where
X is the sample mean.
Note that
Var(X
1
) =
2(1 )
2
+
4
2
4
(2 )
2
=
4 2
2
4
(2 )
2
,
= 2(1
1
) = g(),
g
() = 2/
2
= 2/[2/(2 )]
2
= (2 )
2
/2.
By the central limit theorem and -method,
n(
)
d
N
_
0,
(2 )
2
(2
2
2)
2
_
.
The method of moments can also be applied to nonparametric problems.
Consider, for example, the estimation of the central moments
c
j
= E(X
1
1
)
j
, j = 2, ..., k.
Since
c
j
=
j
t=0
_
j
t
_
(
1
)
t
jt
,
the moment estimator of c
j
is
c
j
=
j
t=0
_
j
t
_
(
X)
t
jt
,
where
0
= 1.
It can be shown (exercise) that
c
j
=
1
n
n
i=1
(X
i
X)
j
, j = 2, ..., k, (2)
which are sample central moments.
From the SLLN, c
j
s are strongly consistent.
If E|X
1
|
2k
< , then
n( c
2
c
2
, ..., c
k
c
k
)
d
N
k1
(0, D) (3)
where the (i, j)th element of the (k 1) (k 1) matrix D is
c
i+j+2
c
i+1
c
j+1
(i + 1)c
i
c
j+2
(j + 1)c
i+2
c
j
+ (i + 1)(j + 1)c
i
c
j
c
2
.
4