7: THE NEYMAN-PEARSON LEMMA

Suppose we are testing a simple null hypothesis

H 0 : = against a simple alternative H 1 : = ,
where is the parameter of interest, and , are
particular values of . We are given a random sample (X 1 , . . . , Xn ) which are iid , each with the
p.d.f. f (x ;).
A p.d.f. for a random variable X , as defined by
Hogg and Craig, p. 39, is either the probability density function (if X is a continuous random variable)
or the probability mass function f (x ) = Pr (X = x )
(if X is a discrete random variable). This definition
is not the standard one, however, as the term p.d.f.

-2is usually reserved for the density of a continuous

random variable. Also note that Hogg and Craig
are assuming that X is either discrete or continuous,
even though there are other possibilities.
We are going to reject H 0 if (X 1 , . . . , Xn ) C ,
where C is a region of the n -dimensional sample
space called the critical region. This specifies a
test. We say that the critical region C has size if
the probability of a Type I error is :
Pr [(X 1 , . . . , Xn ) C ; H 0] = .
We call C a best critical region of size if it has
size , and
Pr [(X 1 , . . . , Xn ) C ; H 1] Pr [(X 1 , . . . , Xn ) A ; H 1]

-3for every subset A of the sample space for which

Pr [(X 1 , . . . , Xn ) A ; H 0] = . Thus, the power
of the test associated with the best critical region C
is at least as great as the power of the test associated with any other critical region A of size .

The Neyman-Pearson Lemma provides us with a

way of finding a best critical region.

The joint p.d.f. of X 1 , . . . , Xn , evaluated at the

observed values x 1 , . . . , xn is called the likelihood function,
L () = f (x 1;) f (x 2;) . . . f (xn ;) .
We often think of L () as a function of alone,

-4although it clearly depends on the data as well.

Define the likelihood ratio as L ()/L ().
Informally, we can think of this as measuring the
plausibility of H 0 relative to H 1. Therefore, if the
likelihood ratio is sufficiently small, we might be
inclined to reject H 0. Example 1, p. 396 of Hogg
and Craig shows that for a binomial random variable with n = 5, the best critical region for testing a
simple null versus a simple alternative involving the
probability of success is the one for which
L ()/L () k , where k is some constant chosen
to ensure that the test has level . The NeymanPearson Lemma asserts that, in general a best criti-

-5cal region can be found by finding the n dimensional points in the sample space for which
the likelihood ratio is smaller than some constant.

The Neyman-Pearson Lemma: If k > 0 and C is a

subset of the sample space such that
L ()/L () k for all (x 1 , . . . , xn ) C

(a)

L ()/L () k for all (x 1 , . . . , xn ) C * (b)

= Pr [(X 1 , X 2 , . . . , Xn ) C ; H 0]

(c)

where C * is the complement of C , then C is a best

critical region of size for testing the simple
hypothesis H 0 : = against the alternative simple
hypothesis H 1 : = .

-6Proof: Suppose for simplicity that the random variables X 1 , . . . , Xn are continuous. (If they were
discrete, the proof would be the same, except that
integrals

would

be

replaced

by

sums).

X = (X 1 , . . . , Xn ). For any region R

Let

of n -

dimensional space, we will denote the probability

that X R by L (), where theta is the true value
R

of the parameter. The full notation, omitted to save

space, would be
Pr [X R ; ] = . . . L ( ; x 1 , . . . , xn ) dx 1 . . . dxn .
R

We need to prove that if A is another critical region

of size , then the power of the test associated with
C is at least as great as the power of the test

-7associated with A , or in the present notation, that

L () L () .

(1)

Suppose X A * C . Then X C , so by (a),

1
L () hh
k

A* C

L () .

(2)

A* C

Next, suppose X A C * . Then X C * , so by

(b),

A C

1
L () hh
k
*

L () .

(3)

A C*

We now establish (1), thereby completing the proof.

L () =

R
J
Q

A C

L ()
J
Q

L () J +

L ()

A C*

L () J +

A* C

A C*

L ()

-81
L () hh
k
C
R

1
L () + hh
k

A* C

L ()

(See (2),(3))

A C*

H
1
1
hh
L () +
L () J
J

k A C
k A C
Q
P
hh

1
1
= L () hh L () + hh L ()
kC
kA
C

(Add Zero )

(Collect Terms )

= L () hh + hh
k
k
C
(Since both C and A have size )
= L () .
C

-9Eg: Suppose X 1 , . . . , Xn are iid N ( , 1), and we

want to test H 0 : = versus H 1 : = , where
> . According to the z -test, we should reject
H 0 if Z = dnd (Xd ) is large, or equivalently if Xd
is large. We can now use the Neyman-Pearson
Lemma to show that the z -test is best. The likelihood function is
n
n
/2
L () = (2)
exp {

(xi )2/2}

i =1

According to the Neyman-Pearson Lemma, a best

critical region is given by the set of (x 1 , . . . , xn )
such that L ()/L () k 1, or equivalently, such
that

1
log [L ()/L ()] k 2. But
n

hh

- 10 1
1 n
hh
log [L ()/L ()] = [(xi )2/2 (xi )2/2]
n
n i =1

hh

1 n
= hhh [ ( xi 2 2xi + 2 ) ( xi 2 2xi + 2 ) ]
2n i =1
1 n
=
[2( ) xi + 2 2]

2n i =1
hhh

1
= ( ) xd + hh [2 2] .
2
So the best test rejects H 0 when xd k , where k is a
constant. But this is exactly the form of the rejection region for the z -test. Therefore, the z -test is
best.