THE T TEST FOR TWO

INDEPENDENT SAMPLES
Chapter 10
GOAL OF HYPOTHESIS TESTING using
t-test

Determine if two or more means are:

1) significantly different from one another OR

2) the difference is due to sampling error

When is t-test appropriate?
• Independent variable: ONE categorical (nominal)
variable with TWO (and only two) categories or possible
values.

• Dependent variable: ONE numeric (interval or ratio)

variable
 Mean 1 Mean 2 # Groups
(samples)

Single-sample Hypothesized Sample 1 Mean 1

T-test Population (post-treatment)
Mean
(Pop 1) (treatment 1)
Independent-samples t-test Sample 1 Mean Sample 2 Mean 2
(post-treatment) (post-treatment)
(treatment 1) (treatment 2)

Repeated measures t-test Sample 1 Mean Sample 1 Mean 1

(related-samples t-test) (pre-treatment) (post-treatment)

1) PRE-POST treatment 1
Sample 1 Mean Sample 1 Mean 1
2) 2 Treatments (post-treatment) (post-treatment)

(treatment 1) (treatment 2)
What means can we compare?
①Sample mean and hypothesized population mean (1 sample of
participants; 1 sample mean)- SINGLE SAMPLE T-TEST

② One mean each from two different samples (2 samples; 2 sample

means)- INDEPENDENT SAMPLE T-TEST

a) Two groups: IV is a Subject variable with 2 categories: variable

that cannot be manipulated; inherent characteristic of participant
 Example: gender- men vs. women’s mean aggression scores

b. Two treatment conditions: IV is an Experimental variable with 2

categories: variable that is manipulated by researcher
 Example: Treated vs. untreated group’s mean anxiety scores
(experimental group and control group)
 Example: Antidepressant vs. psychotherapy group’s mean
depression scores (experimental group vs. comparison group)
What means can we compare?
③ Two means from one sample (1 sample, 2 sample means)-
DEPENDENT OR RELATED SAMPLES T-TEST

a) One treatment: the 1 sample receives 1 treatment and compare

mean scores of sample before and after treatment (PRE-POST
STUDY)
 Example: compare mean anxiety scores both before and after
peer support in stats class

b. Two treatments: the 1 sample receives 2 different treatments and

compare mean scores after each treatment

 Example: compare mean GPA of students AFTER peer reading

support (Treatment 1: given for 3 months) and AFTER addition of
TA to classroom (Treatment 2: given for 3 months)
 PREVIEW
• Katona (1940) tested two different learning methods
1. Memorization
2. Learning by understanding (trial and error)
• METHOD: Participant learned the “match stick problem”/ 12 problems
• DV: After 3 weeks he tested which participants retained information
better (how many problems they solved in 5 minutes)

• QUESTION: Is there a difference between the two styles of learning?

• PROBLEM:
• Our previous t-test only considered one sample mean (single-sample t-test)
• This example has two sample means…

• SOLUTION:
• Use Independent Samples t-test

• Tests TWO SAMPLE MEANS, ONE EACH from

TWO GROUPS OF PARTICIPANTS
Match stick problem – 5 into 4:
• 3 different solutions
PREVIEW: CRITICAL THINKING Q
• In previous example, Katona chose to randomly assign
each participant to 1 of 2 different groups:
①memorization
②learning by understanding

 Why didn’t he just have ONE group of participants take

part in each treatment or experimental group?

 In other words why not have them first use memorization

and then use learning by understanding on matchstick
task?
I. INDEPENDENT MEASURES
DESIGN
 The independent-measures design (or between-subjects
design) allows researchers to evaluate the mean difference
between two populations using the data from two
separate samples.

The IDENTIFYING CHARACTERISTIC of this design is the

existence of two separate or independent samples.

 Thus, an independent-measures design can be used to test for mean

differences between:
1) two distinct groups of people (such as men versus women)
OR
2) two different samples randomly assigned to two treatment
conditions (such as drug versus no-drug)
vs. RELATED SAMPLE DESIGN
 The related-measures design (or within-subjects design)
allows researchers to evaluate the mean difference FOR
ONE population that participates in both treatment
groups.
In this case the means are NOT independent of one another
because the same people are participating in both groups and thus
have the same characteristics in both conditions.

 Thus, a within-subjects design can be used to test for mean

differences between:
1) One sample tested before and after ONE treatment (pre-post)

2) One sample participating in TWO treatment conditions (such as

drug versus no-drug)
BETWEEN-SUBJECTS vs. WITHIN-
SUBJECTS DESIGN
BETWEEN: DIFFERENT SAMPLES WITHIN: SAME SAMPLE

In both designs -> There are 2 treatment conditions.

DV = how well you do on exam out of 100

INDEPENDENT-MEASURES DESIGN
INDEPENDENT MEASURES DESIGN
The independent-measures design is used in situations where a
researcher has no prior knowledge about either of the two
populations (or treatments) being compared.

• In particular, the population means and standard

deviations are all unknown.

Because the population standard deviations are not known,

these values must be estimated from the sample data.
• Just like we did for the single-sample t-test.
II. HYPOTHESIS TESTING WITH T-
STATISTIC
• General purpose of the independent-measures t test is to
determine whether the difference between the two sample
means indicates one of the following:

1) real mean difference between the two populations (or treatments)

1) the obtained difference is simply the result of sampling error.

• Remember: if two samples are taken

from the same population
and are given exactly the same
treatment, there still will be
some difference between
the sample means.
STEPS OF HYPOTHESIS TESTING
Same 4 steps as discussed in chapters on z-score and single
sample-t

1)State the hypotheses and select an α level.

a) -H0 states that there is no difference between the two population means.
-H0 states that one mean less than (greater than) or equal to other mean.
• H0: µ1 - µ2 = 0 or µ1 = µ2 (non-directional null)
• H0: µ1 > µ2 = 0 or µ1 < µ2 (one-directional null)

b) H1 states that there is a difference between the two population means

H1 states that one mean is greater (less than) other mean.
• H1: µ1 - µ2 ≠ 0 or µ1 ≠ µ2 (non-directional alternative)
• H1: µ1 >= µ2 or µ1 <= µ2 (one-directional alternative)
STEPS OF HYPOTHESIS TESTING
(cont.)

2. Locate the critical region using dfT = df1 + df2.

• The critical values for the t statistic are obtained using degrees
of freedom that are determined by adding together the df value
for the first sample and the df value for the second sample.

• df1 = n1 – 1

• df2 = n2 – 1
 STEPS OF HYPOTHESIS TESTING (cont.)

3. Compute the test statistic. (see next slide)

• The t statistic for the independent-measures design has the
SAME BASIC STRUCTURE structure as the single-sample t
(Ch 9).

• However, in the independent-measures design, all components

of the t formula are DOUBLED

There are:
• two sample means (M1 and M2)
• two population means (u1 and u2)
• two sources of error contributing to the standard error in the
denominator
T-FORMULA
One sample t test
Sample mean – Hypo pop mean M  μ (null)
t = ───────────── = ─────
estimated standard error sM
Null = 0
Independent sample t test
Sample mean diff – No pop mean diff M1-M2  (μ1-µ2)
t = ───────────── = ─────
estimated standard error s(M1-M2)

Estimated Standard error

• How much difference is reasonable to expect between the two sample means if the null
hypothesis is true

Actual mean difference (M1 – M2)- 0 sample data- null hypo

___________
indep t = ────────────────
estimated standard error
= error
 STEPS OF HYPOTHESIS TESTING (cont.)

4. Make a decision.

• If difference between 2 sample means (M1 - M2: numerator)

is > than the

difference expected by chance (SEM; denominator), then . . .

REJECT H0 (NULL)

(& conclude that there is a REAL MEAN DIFFERENCE

between the 2 populations or treatments)
 III. CALCULATING THE ESTIMATED SEM
But how do you calculate the eSEM with two samples? (SM1-M2)

eSEM for single-sample t: s 2 = Sample variance

sm =
n

Each sample mean represents its own population mean

• M1 approximates µ1 with some error
• M2 approximates µ2 with some error
• Two sources for error

= Sample 2 variance
s12 = Sample 1 variance s22
M1sM = M 2 sM =
n1 n2
A. ESTIMATED SEM: FORMULA

 Unlike the numerator where you subtract the means for each
sample (to find mean difference),
in the denominator, you add the variances from each sample.

S(M1-M2) = s1/sqrt(n) + s2/sqrt(n)

This formula ONLY applies to cases where the two

samples have equally sample sizes.
A. ESTIMATED SEM: FORMULA (cont.)
 In cases where the samples have different sizes (n), this
formula would provide a biased estimate.

S(M1-M2) = s1/sqrt(n) + s2/sqrt(n)

WHY?
 Because samples with larger sample sizes provide better estimates of
the population variance (lower sampling error) than those with a
small sample size. So need to WEIGHT each sample accordingly.

In order to compensate for this we take the:

 pooled variance: AVERAGING of the two sample variances,
which allows for the larger variance to carry more weight
B. POOLED VARIANCE (EQUAL SAMPLE SIZES)

One sample

 To calculate the POOLED VARIANCE for samples with the same size,
we use the formula (with the sum of squares) below
B. POOLED VARIANCE (DIFFERENT SAMPLE SIZES)

To calculate the POOLED VARIANCE for samples with

the different sizes, we use the formula below.

 Just as with weighted mean, each variance must be multiplied by its

sample size or, in this case, degrees of freedom.

 Provides unbiased estimate when have unequal sample sizes

C. ESTIMATED SEM: FINAL FORMULA

 You can then use the Pooled Variance in place of the

individual variance of each sample to calculate an Estimated
SEM that takes into account the size of each sample.

 ESTIMATED SEM for either equal or unequal sample sizes (UNBIASED)

s s 2
p
2
p
s(M 1 -M 2 ) = +
n1 n 2
D. FINAL INDEPENDENT SAMPLES
T-STATISTIC FORMULA
IV. INTERPRETING T-STATISTIC AND DEGREES
OF FREEDOM
• Degrees of freedom: the degrees of freedom for the independent
measures t statistic are determined by the df values for the two
separate samples

df for the t statistic = df for the 1st sample + df for the 2nd sample
=df1 +df2
=(n1-1)+(n2-1) OR (n1+ n2 - 2)
EXAMPLE: INDEPENDENT SAMPLES
T-TEST
 TV viewing habits and future performance in high school

• HYPO: There will be a difference in the mean high school GPA between
hildren who watched Sesame Street (group 1) and children who did not
watch Sesame Street (group 2)

• TWO GROUPS: n=20; 10 who did not watch & 10 who did watch

MEAN HIGH SCHOOL GRADE

Did watch Did not watch
86 99 90 79
87 97 89 83
91 94 82 86
97 89 83 81
98 92 85 92
n=10 n=10
M=93 M=85
SS=200 SS=160
 EXAMPLE: 4 STEPS OF
HYPOTHESIS TESTING
 Step 1 – State hypothesis and select α = .05
 Ho: µ1 = µ2
 H1: µ1 ≠ µ2

 Step 2 – Locate the critical region.

a) df= df1+ df2
• (n1-1) + (n2-1)
• =9+9
• =18

b) t > 2.101 or < -2.101

(from table)
EXAMPLE: 4 STEPS OF
HYPOTHESIS TESTING (cont.)
• Step 3 – compute t-statistic – in three parts

1) Pool the variances for both samples

SS1 + SS 2 200 +160 360
sp =
2 sp 2 = = = 20
df1 + df 2 9 +9 18

2) Calculate the total Estimated Standard Error

s2p s2p 20 20
s(M 1 -M 2 ) = + = + = 2+2 = 4 = 2
n1 n2 10 10
3) Compute the t
(M1 - M 2 ) - (m1 - m 2 ) (93 - 85) - 0 8
t= = = =4
s(M 1 -M 2 ) 2 2
EXAMPLE: 4 STEPS OF
HYPOTHESIS TESTING (cont.)
• Step 4 – Make a decision
• t= + 4.0

Does watching Sesame Street as a five-year old affect high school

performance?
 (if positive) Is calculated t > Critical t? +4.0 > +2.101

 YES it is! So do we
Retain or reject null?

 REJECT THE NULL

EXAMPLE: APA REPORTING
 How do we write it in APA style?

 According to an independent two-samples t-test, children

who watched Sesame Street had a significantly different
GPA in high school (M = 93, SD = 3.8) than children who
did not watch Sesame Street (M = 85, SD = 2.7), t(18) =
4.00, p < .05.
V. OTHER TOPICS
A. EFFECT SIZE  How do you calculate effect size?

Average High School Grade  Compute Cohen’s d!

Watched Did not watch

Sesame street Sesame street M1 - M 2 93-85
d= = =1.79
sp2
20
86 99 90 79
87 97 89 83
91 94 82 86 Cohen’s d Effect Size
97 89 83 81 d = 0.2 Small effect
98 92 85 92 d = 0.5 Medium effect
n = 10 n = 10 d = 0.8 Large effect
M = 93 M = 85
SS = 200 SS = 160
B. INFLUENCE OF SAMPLE
VARIANCE ON T-TEST
• High variability reduces the chances of obtaining
significance

 Example:
• Two groups- both n=9 with a 5 point mean difference
• Group 1: M=8, s=1.22 Group 2: M=13, s=1.22—SMALL amount of variability
• No overlap between groups
• Easily reject the null!
• α .05 or .01

mean difference 5
t= = = 8.62
estimated stan dard error .58
 INFLUENCE OF SAMPLE VARIANCE
ON T-TEST (cont.)
• Increasing the sample variance causes scores to be more
spread out.

 Example:
 Group 1: n=9, M=8, Group 2: n=9, M=13 (same M and n)

• But the VARIANCE is increased

Group 1: s=6.65 Group 2: s=6.65
• There is a lack of clear distinction between groups

mean difference 5
t= = =1.59
estimated stan dard error 3.14
C. ASSUMPTIONS FOR T-TEST
• There are three assumptions that must be met when
performing an independent t test:

① The observations must be INDEPENDENT of each other

(INDEPENDENT SAMPLES)

② The TWO POPULATIONS that the samples are taken from

must be NORMALLY DISTRIBUTED

③ The TWO POPULATIONS that the samples are chosen from

must have EQUAL VARIANCES (NOT EQUAL SAMPLE
SIZES)
ASSUMPTIONS FOR T-TEST (cont.)
 Although most hypothesis tests are built on a set of underlying
assumptions, the tests usually work reasonably well even if
the assumptions are violated.

The one notable exception is the assumption of homogeneity of

variance for the independent-measures t test.

 Homogeneity of variances test: determines if two populations

from which the samples are obtained have equal variances.

 Most common: Levene’s test

Alternative: F-Max test

HOMOGENEITY OF VARIANCE
1) Fmax test = = Larger variance/ smaller variance (calculated F-
max)

2) Critical Fmax (look Appendix- table B.3)

K = # of separate samples (k= 2)
df = n-1 (all samples must be equal sample size)

3) Is calculated Fmax > Critical Fmax?

****Then homogeneity of variances assumption is violated!

 Mean 1 Mean 2 # Groups
(samples)

Single-sample Hypothesized Sample 1 Mean 1

T-test Population (post-treatment)
Mean
(Pop 1) (treatment 1)
Independent-samples t-test Sample 1 Mean Sample 2 Mean 2
(post-treatment) (post-treatment)
(treatment 1) (treatment 2)

Repeated measures t-test Sample 1 Mean Sample 1 Mean 1

(related-samples t-test) (pre-treatment) (post-treatment)

1) PRE-POST treatment 1
Sample 1 Mean Sample 1 Mean 1
2) 2 Treatments (post-treatment) (post-treatment)

(treatment 1) (treatment 2)
TERMS
• Independent measures
• Between subjects design
• Within subjects design
• Repeated measures
• Estimated standard error
• Pooled variance
• Homogeneity of variance
• F-Max