Hypothesis Testing

09/07/17

1. Methods of Finding Tests

1.1 Likelihood Ratio Tests

Recall that if " , . . . , % be a random sample from a population with pdf or pmf | ,
then the likelihood function is defined as

|" , . . . , % = | = | = - | .
-."

Definition 1.1.1 (Likelihood Ratio Test, LRT)

Let denote the entire parameter space for , then the likelihood ratio test statistic for
testing 1 : 1 vs " : 41 is

sup9:; |
=
sup9: |
1 |
= ,
|

where 1 is the MLE of obtained from the null hypothesis set 1 , and is the MLE
of obtained from entire set = 1 41 . A likelihood ratio test (LRT) is any test that
has a rejection region of the form

where is any number satisfying 0 1.

Example 1.1.2 (Normal LRT)
Let " , . . . , % be a random sample from a , 1 population, and consider testing
1 : = 1 vs " : 1 . Then, the LRT statistic is

1 |
=
|
%
G % H
2 H exp -." - 1 /2
= %
G % H
2 H exp -." - /2
% %
H H
= exp - 1 - /2
-." -."
H
= exp 1 /2 .

The rejection region is

exp 1 H /2

1 2 log / .

Theorem 1.1.3
If is a sufficient statistic for and and are the LRT statistics based
on and , respectively, then = for every in the sample space.

Example 1.1.4 (Normal LRT and Sufficiency)

"
Recall Example 1.1.2. We know that is a sufficient statistic for and ~ , .
%

By Theorem 1.1.3, the LRT statistic is

1 |
=
|
"
G
2/ H exp 1 H /2
= "
G H /2
2/ H exp
= exp 1 H /2 .
1.2 Union-Intersection and Intersection-Union Tests
The union-intersection test (UIT) is useful when the null hypothesis set is expressed as
Y
an intersection, 1 = X." X . Then the hypotheses are expressed as

Y Y

1 : X vs " : X4 .
X." X."

Let X X denote a rejection region for a test of 1X : X vs "X : X4 .

Then the rejection region for the UIT is

X X .
X."

The intersection-union tests (IUT) is useful when we consider problems in which the null
Y
hypothesis set is expressed as a union, 1 = X." X . Then the hypotheses are expressed
as

Y Y

1 : X vs " : X4 .
X." X."

Let X X denote a rejection region for a test of 1X : X vs "X : X4 .

Then the rejection region for the IUT is

X X .
X."
Example 1.2.1 (Normal UIT)
Let " , . . . , % be a random sample from a , H population, H known. Consider
testing 1 : = 1 vs " : 1 . We can write 1 as the intersection of two sets,

1 : 1 1 .

First, the LRT statistic of 1_ : 1 vs "_ : > 1 is, for > 1 ,

1 |
_ = = exp 1 H /2 H ,
|

and, for 1 , _ = 1. The rejection region, with > 1 , is

_ exp 1 H /2 H
1 / / 2 log
1 / / a .

Next, the LRT statistic of 1b : 1 vs "b : < 1 is, for < 1 ,

1 |
b = = exp 1 H /2 H ,
|

and, for 1 , b = 1. The rejection region, with < 1 , is

b exp 1 H /2 H
1 / / 2 log
1 / / aa .

Finally, the rejection region for the IUT is

1 / / a 1 / / aa .
2. Methods of Evaluating Tests

2.1 Error probabilities and the Power Function

Consider testing 1 : 1 vs " : 41 . We might make two types of errors:

Decision
Accept 1 Reject 1
Correct Type I
1
Decision Error
Truth
Type II Correct
"
Error Decision

Let denote the rejection region for a test, so we reject 1 : 1 if . Then,

Type I Error if 1
9 =
1 Type II Error if 41 .

Definition 2.1.1 (Power Function)

The power function of a hypothesis test with rejection region is the function of
defined by = 9 .

A good test has power function near 1 for most 41 and near 0 for most 1 .
Example 2.1.2 (Normal Power Function)
Let " , . . . , % be iid , H population with H known, and we consider following
test: 1 : 1 vs " : > 1 . The LRT statistics is, for > 1 ,

1 |
= = exp 1 H /2 H ,
|

and, for 1 , = 1. The rejection region, with > 1 , is

= exp 1 H /2 H
1 / / 2 log
1 / / a .

The power function of this test is

1 1
= a = a +
/ / /
1
= a + ,
/

where is a standard normal random variable.

For a fixed , it is usually impossible to make both types of error probabilities arbitrarily
small. In searching for a good test, it is common to control the Type I Error probability at
a specified level.

Definition 2.1.4 (Size of Tests)

For 0 1, a test with is a size test if sup9:; = .

Definition 2.1.5 (Level of Tests)

For 0 1, a test with is a level test if sup9:; .
Example 2.1.6 (Size of Normal LRT)
A size LRT is constructed by choosing such that

sup9:; 9 = .

"
Recall Example 1.1.2. Since 1 consists of 1 and ~ 1 , under 1 ,
%

sup9:; 9 = 1 2 log /

1
= 2 log

= || 2 log
= .

Take 2 log = q/H , where q/H satisfies q/H = /2, then this test is the
H
size LRT with = exp q/H /2 .

Definition 2.1.7 (Unbiased Test)

A test with is unbiased if a aa for every a 41 and aa 1 .

Example 2.1.8 (Normal Unbiased LRT)

Recall Example 2.1.2. The power function is

1
= a + .
/

Since is an increasing function of for fixed 1 , it follows that

> 1 = maxtuv; ,

for all > 1 . Hence this test is unbiased.

2.2 Most Powerful Tests
Some of classes of hypothesis tests control the probability of a Type I Error, for example,
the level test. A good test would also have a small Type II Error probability.

Definition 2.2.1 (Uniformly Most Powerful Test, UMP Test)

Let be a class of tests for testing 1 : 1 vs " : 41 . A test in , with power
function , is a uniformly most powerful (UMP) class test if a for
every 41 and every a that is a power function of a test in .

In this section, the class will be the class of all level tests. Then, the test described
in Definition 2.2.1 is called a UMP level test.

Theorem 2.2.2 (Neyman-Pearson Lemma)

Consider testing 1 : = 1 vs " : = " , where the pdf of pmf corresponding to -
is |- , = 0, 1, using a test with rejection region that satisfies

|"
if >
|1
and (1)
|"
4 if < ,
|1

for some 0, and

= 9; . (2)

Then
a. (Sufficiency) Any test that satisfies (1) and (2) is a UMP level test.
b. (Necessity) If there exists a test satisfying (1) and (2) with > 0, then every UMP
level test is a size test and every UMP level test satisfies (1) except perhaps on
a set satisfying 9; = 9{ = 0.
Corollary 2.2.3
Suppose is a sufficient statistic for and |- is the pdf or pmf of
corresponding to - , = 0, 1. Then any test based on with rejection region is
a UMP level test if it satisfies

|"
if >
|1
and (3)
|"
4 if < ,
|1

for some 0, where

= 9; .

Example 2.2.4 (UMP Normal Test)

Let " , . . . , % be iid , H , H known. We know that is a sufficient statistic for
. Consider testing 1 : = 1 vs " : = " , where 1 > " . With " 1 < 0, the
inequality (3) in Corollary 2.2.3 is equivalent to

|" 2 H / G"/H
exp " H /2 H
> >
|1 2 H / G"/H exp H /2 H
1

2 H log / 1H + "H
< = .
2 " 1

Under 1 , ~ 1 , . Thus
%

1 1 1
v; < = < = < .
/ / /

Take = 1 q / , then v; < = . Therefore, by Corollary 2.2.3, the test

with rejection region < is the UMP level test.
Definition 2.2.5 (Monotone Likelihood Raito, MLR)
A family of pdfs of pmfs | for a univariate random variable with
real-valued parameter has a monotone likelihood ratio (MLR) if, for every H > " ,
|H / |" is a monotone (nonincreasing or nondecreasing) function of on
|" > 0 or |H > 0 . Note that /0 is defined as if > 0.

Example 2.2.6 (MLR Property of the Normal Family)

Consider , H with H known. The likelihood ratio for H > " ,

|H 2 H G"/H
exp H H /2 H
=
|" 2 H G"/H exp H /2 H
"
H H
= exp H " /2 H
= exp H " / H exp "H HH /2 H .

Since H " > 0, the ratio is increasing in . So, the family of , H have MLR.

Theorem 2.2.7 (Karlin-Rubin)

Consider testing 1 : 1 vs " : > " . Suppose that is a sufficient statistic for
and the family of pdfs or pmfs | of has an MLR property. Then
for any 1 , the test that rejects 1 if and only if > 1 is a UMP level test, where
= 9; > 1 . Under the same conditions, the test that rejects 1 : 1 in favor of
" : < " if and only if if < 1 is a UMP level test, where = 9; < 1 .

Example 2.2.8
Recall Example 2.2.4. Consider testing 1a : 1 vs "a : < 1 using the test that
rejects 1a if

< 1 q / .

As is a sufficient statistic for and its distribution has MLR property, it follows from
Theorem 2.2.7 that the test is a UMP level test.
3. Asymptotic Properties

3.1 Asymptotic Distribution of LRTs

If we cannot derive a simple formula for for , we can get an approximate answer.

Theorem 3.1.1 (Asymptotic Distribution of the LRT simple )

Consider testing 1 : = 1 vs " : 1 . Suppose " , . . . , % are iid | , is
the MLE of , and | satisfies the regularity conditions in p.516. Then under 1 ,
as ,

2 log "H ,

where "H is a chi-squared random variable with 1 degree of freedom.

Example 3.1.2 (Poisson LRT)

For testing 1 : = 1 vs " : 1 based on " , . . . , % be iid Poisson , we have

exp 1 1 { 1
2 log = 2 log = 2 1 log ,
exp {

%
where = -." - / is the MLE of . By Theorem 3.1.1, we would reject 1 at level
H
if 2 log > ",q .

Theorem 3.1.3
Let " , . . . , % be a random sample from a pdf or pmf | , and consider testing
1 : 1 vs " : 14 . Assuming the regularity conditions in p.516, under 1 ,

2 log H ,

as , where is the difference between the number of free parameters specified

by 1 and the number of free parameters specified by .
Example 3.1.4 (Multinomial LRT)
Let " , . . . , % be iid discrete random variables with - = = X , = 1, . . . , 5, with

X." X = 1 and X 0. With = " , H , , , , the likelihood function is

%

| = - | = " { H ,
-."

where X is number of " , . . . , % equal to . Consider testing

1 : " = H = and = vs " : 1 is not true.

Since = 1 " H , there are only 4 free parameters in . There is only

"G {
1 free parameter in 1 . Once " is fixed, H = = " and = = . Thus,
H

the degrees of freedom is = 3. The MLE of under is X = . Under 1 ,
%

1 3"
| = " { .
2

{ "G {;
The MLE of under 1 is "1 = H1 = 1 = and 1 = 1 = .
% H

Thus, the test statistic is

{
" + H + " + H + " + H + + +
=
3" 3H 3 2 2

and

-
2 log = 2 - log ,
-
-."

{
where " = H = = and = = .
H