40.

004: Statistics
Lecture 9

I NFERENCE (C HAPTER 7)
S INGLE SAMPLE TESTS , P - VALUES AND POWERS

Term 5, 2018

Das, Kushnarev 40.004: Statistics Term 5, 2018 1 / 16

HW and exam

Homework 2 is available now. Homework 1 solutions are on eDimension.

Exam 1:

Wednesday 28th March, 2:30-4:30pm.

Venue: TT 24, TT 25, CC 14: check e-dimension for your room.

Everything taught up to Chapter 8, inference for two-samples (comparing means) will be

examinable. This includes material taught until Thursdays class (22 Feb) class.

A non-programmable, scientific calculator is allowed; one A4 sheet with handwritten notes

on both sides is also allowed.

A normal distribution table, as well as all required values from other distributions will be
provided.

Das, Kushnarev 40.004: Statistics Term 5, 2018 2 / 16

Chapters from the book
Statistics and Data Analysis, Tamhane and Dunlop

Lectures 1-10 cover material for Exam 1

Lecture 1: Chapter 1 and additional material (some class notes)
Lecture 2: Chapter 3
Lecture 3: Chapter 4: sections 4.1, 4.2, 4.3
Lecture 4: Chapter 4: sections 4.3, 4.4, 4.5 (skip 4.5.2)
Lecture 5: Chapter 5: section 5.1, German tank problem
Lecture 6: Chapter 5: sections 5.2, 5.3
Lecture 7: Chapter 6: sections 6.1 (skip 6.1.2), 6.2
Lecture 8: Chapter 6: section 6.3 (skip 6.3.4)
Lecture 9: Chapter 7: sections 7.1, 7.2, 7.3
Lecture 10: Chapter 8: sections 8.1, 8.2, 8.3
The following are not in the syllabus of Exam 1
Lecture 11: Chapter 8: Section 8.4
Lecture 12: Chapter 9

Das, Kushnarev 40.004: Statistics Term 5, 2018 3 / 16

Hypothesis Testing

- H0 : Null hypothesis (in general, status quo, hypothesis we are seeking evidence against).
- HA : Alternative hypothesis (new belief, hypothesis which requires evidence for)

Conduct experiment, collect data and propose a test statistic.

Create a decision criterion based on the test statistic.

How likely is the value of the test statistic under H0 ?

Type I error: Rejecting H0 when H0 is true.

Type II error: Not rejecting H0 when HA is true.

Das, Kushnarev 40.004: Statistics Term 5, 2018 4 / 16

Hypothesis Testing

Level of significance of a test is α if Pr(Type I error) ≤ α.

Power of a test is 1 − β where β = Pr(Type II error).

Ideally, a test should have low level and high power. It is not easy to achieve both unless
sample size is increased (more on this later).

Instead of fixing a level α, we often report the p-value, which is the probability that under
H0 , a test statistic value as extreme as observed or worse is realised.

A level-α test rejects H0 if the p-value ≤ α.

The lower the p-value, the more significant is our finding.

Das, Kushnarev 40.004: Statistics Term 5, 2018 5 / 16

Hypotheses testing for means
Suppose we observe samples from a population with mean µ. The types of hypotheses that may
be of interest to us are:

1 H0 : µ = µ0 vs. HA : µ 6= µ0 .
2 H0 : µ = µ0 vs. HA : µ = µ1 .
3 H0 : µ = µ0 vs. HA : µ > µ0 .
4 H0 : µ ≤ µ0 vs. HA : µ > µ0 .

For a one-sided alternative hypothesis, such as HA : µ > µ0 , it does not matter if we use

H0 : µ = µ0 or H0 : µ ≤ µ0 ,
for computing p-value of the test-statistic.
If we use the latter, then the maximum p-value is still obtained at the boundary, when
µ = µ0 .

One-sided tests are used when the deviation is expected to be in a particular direction.
They should not be used as a device to make a statistically non-significant result
significant.

Das, Kushnarev 40.004: Statistics Term 5, 2018 6 / 16

Hypothesis test: a summary

For large samples (n > 30) we assume sample variance s2 ≈ σ 2 and we can use CLT.

A few equivalent ways to perform a hypothesis test for the mean µ (with σ known).

Calculate the appropriate (1 − α)-level confidence interval around x, and check if µ0

falls outside it.

Calculate the appropriate p-value and compare it with α.

x−µ
√ 0 , and compare it with the appropriate critical value
Calculate the z-statistic, σ/ n
(which may be zα , or, zα/2 ).

When σ is unknown and we have a small sample, it is customary to assume that the data
is normally distributed and resort to a t-statistic: x−µ
√0 .
s/ n

Das, Kushnarev 40.004: Statistics Term 5, 2018 7 / 16

Hypothesis test: exercise

Exercise: Suppose that you selected a random sample of 36 SUTD students, and found that on
average, they spend 20.0 hours on homework per week, with a sample standard deviation of
3.0 hours (assume normality, you may use CLT with σ approximated accurately by s).

For the hypotheses H0 : µ = 19 vs HA : µ > 19,

1 Find the p-value.

2 Can H0 be rejected if α = 5%? α = 1%?

Das, Kushnarev 40.004: Statistics Term 5, 2018 8 / 16

Power: exercise 1

Exercise: Previous research has shown that the amount of time children spend watching TV per
week had µ = 22.6h and σ = 6.1h. A market research firm believes that the stated mean is now
too low. A random sample of 60 children are taken to measure the number of hours they watch
TV. A hypothesis test at the α = 0.01 level is carried out.

1 State H0 and HA .

2 Can we use the CLT?

3 What is the decision criterion?

4 Suppose the true mean for this population is 25 hours. What is β, and what is the power in
this case? (Draw a picture!)

Das, Kushnarev 40.004: Statistics Term 5, 2018 9 / 16

α and β

Note: α and β cannot be reduced simultaneously, unless we increase the sample size.

https://shiny.rit.albany.edu/stat/betaprob/

Das, Kushnarev 40.004: Statistics Term 5, 2018 10 / 16

Power calculation – formula

Assume that σ is known, and that n is large so we may use the z-distribution.

Consider the problem of testing H0 : µ = µ0 vs HA : µ > µ0 . Then the power (1 − β), as a

function of µ, is given by

 √ 
(µ − µ0 ) n
π(µ) = 1 − β = Φ − zα .
σ

Proof: generalize from Exercise 1. You should also figure out the corresponding formulas
for HA : µ < µ0 and HA : µ 6= µ0 .

Note: in situations where we need to use the t-distribution, the power calculation is less
straightforward.

Das, Kushnarev 40.004: Statistics Term 5, 2018 11 / 16

Sample size determination

We can now relate the required sample size to α and β.

With the assumptions on the previous slides, the minimum sample size required for an
α-level hypothesis test with power of (1 − β) is
 2
(zα + zβ )σ
n= ,
µ − µ0

rounded to the next integer.

Das, Kushnarev 40.004: Statistics Term 5, 2018 12 / 16

Sample size determination

Consider a (1 − α) two-sided confidence interval for µ using the z-distribution. What is the
relationship between the width of the interval and the sample size?

If the width of the CI is 2E, then we require the minimal sample size to be

2
zα/2 σ

n= ,
E

rounded to the next integer.

Exercise: Find the required sample size for a 95% CI, whose width is σ/4.

Das, Kushnarev 40.004: Statistics Term 5, 2018 13 / 16

Power: exercise 2

Exercise: Changes in test scores for students retaking the SAT without coaching has µ = 15
and σ = 40. The changes in the scores are roughly normally distributed. A coaching program
claims that on average it can improve the mean score by at least 35 points. A 0.01-level test of
H0 : µ = 15 vs HA : µ > 15 is to be conducted. Find the number of students that must be tested
in order to have at least 90% power for detecting an increase of 35 points or more.

53

Das, Kushnarev 40.004: Statistics Term 5, 2018 14 / 16