DESIGN OF EXPERIMENTS

INTRODUCTION

Analysis of Variance (ANOVA) is a hypothesis-testing technique used to test the equality of two or
more population (or treatment) means by examining the variances of samples that are taken.
ANOVA allows one to determine whether the differences between the samples are simply due to
random error (sampling errors) or whether there are systematic treatment effects that cause the mean
in one group to differ from the mean in another.

Most of the time ANOVA is used to compare the equality of three or more means, however when the
means from two samples are compared using ANOVA it is equivalent to using a t-test to compare the
means of independent samples.

ANOVA is based on comparing the variance (or variation) between the data samples to variation within
each particular sample. If the between variation is much larger than the within variation, the means of
different samples will not be equal. If the between and within variations are approximately the same
size, then there will be no significant difference between sample means.

ELEMENTS OF A DESIGNED EXPERIMENT

Definition 1
The response variable is the variable of interest to be measured in the experiment. We also refer to
the response as the dependent variable.

Definition 2
Factors are those variables whose effect on the response is of interest to the experimenter.
Quantitative factors are measured on a numerical scale, whereas qualitative factors are not
(naturally) measured on a numerical scale.

Definition 3
Factor levels are the values of the factor utilized in the experiment.

Definition 4
The treatments of an experiment are the factor-level combinations utilized.

Definition 5
An experimental unit is the object on which the response and factors are observed or measured.

Definition 6
A designed experiment is an experiment in which the analyst controls the specification of the
treatments and the method of assigning the experimental units to each treatment. An observational
experiment is an experiment in which the analyst simply observes the treatments and the response on
a sample of experimental units
Definition 7
The completely randomized design is a design in which treatments are randomly assigned to the
experimental units or in which independent random samples of experimental units are selected for
each treatment.

Design of Experiments is classified into three types

1) Completely Randomized Design or One-way Analysis of Variance

2) Randomized Design or Two-way Analysis of Variance
3) Latin Square Design or Three-way Analysis of Variance

1) COMPLETELY RANDOMIZED DESIGN or One-way Analysis of Variance

The test procedure compares the variation in observations between samples to the variation within
samples. Completely randomized designs are the simplest in which the treatments are assigned to the
experimental units completely at random. This allows every experimental unit, i.e., plot, animal, soil
sample, etc., to have an equal probability of receiving a treatment.

Suppose we wish to compare k population means ( k  2 ). This situation can arise in two ways. If the
study is observational, we are obtaining independently drawn samples from k distinct populations and
we wish to compare the population means for some numerical response of interest. If the study is
experimental, then we are using a completely randomized design to obtain our data from k distinct
treatment groups. In a completely randomized design the experimental units are randomly assigned to
one of k treatments and the response value from each unit is obtained. The mean of the numerical
response of interest is then compared across the different treatment groups.

Advantages of Completely Randomized Designs

1. Complete flexibility is allowed - any number of treatments and replicates may be used.
2. Relatively easy statistical analysis, even with variable replicates and variable experimental errors for
different treatments.
3. Analysis remains simple when data are missing.
4. Provides the maximum number of degrees of freedom for error for a given number of experimental
units and treatments.

Disadvantages of Completely Randomized Designs

1. Relatively low accuracy due to lack of restrictions which allows environmental variation to enter
experimental error.
2. Not suited for large numbers of treatments because a relatively large amount of experimental
material is needed which increases the variation.

Appropriate Use of Completely Randomized Designs

1. Under conditions where the experimental material is homogeneous, i.e., laboratory, or growth
chamber experiments.
2. Where a fraction of the experimental units is likely to be destroyed or fail to respond.
3. In small experiments where there is a small number of degrees of freedom.
The completely randomized design is seldom used in field experiments where the randomized
complete block design has been consistently more accurate since there are usually recognizable
sources of environmental variation.

In One way Classification

Null Hypothesis H o : 1  2  ...  k

Alternative Hypothesis: H a : at least two population means differ, i.e. i   j for some i  j.

Assumptions:
1. Samples are drawn independently (completely randomized design)
2. Population variances are equal, i.e.  12   22     k2 .
3. Populations are normally distributed.

Notations:

Number of Samples (or levels) =k

Number of observations in i th sample = ni, i = 1,2,3,…, k

Total Number of observations = n   ni

i
Observation j in the i th sample = xij, j = 1,2,3,….,ni

Sum of ni observations in i th sample = Ti =  x ij

j

Sum of all observations T   Ti   xij

i i j

The Computational Formulae

 T2
Total Sum of Squares SST   x  2
ij
i j n
Ti 2 T 2
Between samples sum of squares, SS B   
i ni n
Within samples sum of squares SSW = SST - SSB

SST SS B
Total mean square MS T  Between samples square MS B 
n 1 k 1
SS
Within sample square, MS W  W Number of d.o.f = (k -1) + (n – k) = n – 1.
nk

ANOVA TABLE
Source of variation Sum ofSquares Degrees of Freedom Mean Square F Ratio
Between samples SSB k-1 MSB MS B
Within samples SSW n-k MSW MS W
Total SST n-1

Working Method

STEP I: Set up Null Hypothesis H 0  1   2       k (Population means are equal)

STEP II: Set up Alternative Hypothesis H1  1   2       k (Population means are not equal)
STEP III: Take l.o.s = α
STEP IV: Find total number of observations n.
STEP V: Calculate T, the Grand Total number of observations.
STEP VI: Calculate the sum of squares SST, SSB, SSW.
STEP VII: Prepare ANOVA Table to calculate F-Ratio.
STEP VIII: Conclusions.
i) If calculated F > Fα for Fα, (k -1) + (n – k) d.o.f , Reject H0
ii) If calculated F < Fα for Fα, (k -1) + (n – k) d.o.f , Accept H0

PROBLEM 1
Neuroscience researchers examined the impact of environment on rat development. Rats were
randomly assigned to be raised in one of the four following test conditions: Impoverished (wire mesh
cage - housed alone), standard (cage with other rats), enriched (cage with other rats and toys), super
enriched (cage with rats and toys changes on a periodic basis). After two months, the rats were tested
on a variety of learning measures (including the number of trials to learn a maze to a three perfect trial
criteria), and several neurological measure (overall cortical weight, degree of dendritic branching, etc.).
The data for the maze task is below. Compute the appropriate test for the data provided below.
Impoverished Standard Enriched Super Enriched
22 17 12 8
19 21 14 7
 15 15 11 10
24 12 9 9
18 19 15 12

Solution:
Source of variation Sum ofSquares Degrees of Freedom Mean Square F Ratio
Between samples SSB= 323.35 3 107.7833 MS B
Within samples SSW = 135.6 16 8.475  12.71
MSW
Total SST = 458.95 19
Null Hypothesis H 0  1   2  3   4
Alternative Hypothesis H1: At least two means differ
Test Statistic: Fc = 12.71
Table Value F0.05,(3,16)= 3.49
Conclusion: Fc > F0.05,(3,12) , Reject Null Hypothesis

1. What is your computed answer? F = 12.71 (3,16) p < .01

2. What would be the null hypothesis in this study? Environment will have no impact on learning
ability as operationalized by maze performance in rats.
3. What would be the alternate hypothesis? Environment will have an impact on learning ability
as operationalized by maze performance in rats.
4. What is your Fcrit? Fcrit = 5.29
5. Are there any significant differences between the four testing conditions? Yes - There is no
significant difference between the impoverished group and the standard group (F comp = 2.32
and qobs= 2.15, n.s.). There is a significant difference between the impoverished group and both
the enriched and supenriched group (Fcomp = 16.15 and qobs= 5.68, p < .01) and Fcomp = 31.90 and
qobs= 7.98, p < .01), respectively). There is no significant difference between the standard group
and the enriched group (Fcomp = 6.24 and qobs= 3.53, n.s.). There is a significant difference
between the standard group and the supenriched group (Fcomp = 17.03 and qobs= 5.83, p < .05).
There is no significant difference between the enriched group and the superenriched group
(Fcomp = 2.65 and qobs= 2.30, p < .05)).
6. Interpret your answer. Environment may have an impact on ability to learn. Differences were
found between groups when each group is compared to a group at least two levels above the
one under study. Thus for example, there is a difference between the impoverished and the
enriched and superenriched but not between the impoverished and the standard groups.

PROBLEM 2

A research study was conducted to examine the clinical efficacy of a new antidepressant. Depressed
patients were randomly assigned to one of three groups: a placebo group, a group that received a low
dose of the drug, and a group that received a moderate dose of the drug. After four weeks of
treatment, the patients completed the Beck Depression Inventory. The higher the score, the more
depressed the patient. The data are presented below. Compute the appropriate test.
 Placebo Low Dose Moderate Dose
38 22 14
47 19 26
39 8 11
25 23 18
42 31 5
Solution:
Source of variation Sum ofSquares Degrees of Freedom Mean Square F Ratio
Between samples SSB= 1484.9333 2 742.46666 MS B
 11.26
Within samples SSW = 790.8 12 65.9 MSW
Total SST = 2275.73333 14

Null Hypothesis H 0  1  2  3
Alternative Hypothesis H1: At least two means differ
Test Statistic: Fc = 11.26
Table Value F0.05,(2,12)= 6.93
Conclusion: Fc > F0.05,(2,12) , Reject Null Hypothesis

1. What is your computed answer? F = 11.26 (2,12) p < .01

2. What would be the null hypothesis in this study? There will be no difference in depression
levels between the three groups. The groups taking the drug will not be different than the
groups taking the placebo.
3. What would be the alternate hypothesis? There will be a difference somewhere in depression
levels between the three levels of drug groups.
4. What probability level did you choose and why? p = .01. There is a risk involved with a Type I
error. I do not want to erroneously say the drug works and then later find out that it doesn't.
5. What is your Fcrit? Fcrit = 6.93
6. Is there a significant difference between the groups? Yes - a significant difference exists
somewhere between the three groups.
7. If there is a significant difference, where specifically are the differences? There is a significant
difference between the placebo group and the low dose group (Fcomp = 11.75 and qobs= 4.84, p <
.05). There is a significant difference between the placebo group and the moderate dose group
(Fcomp = 20.77 and qobs= 6.44, p < .01). There is no significant difference between the low dose
and the moderate dose groups (Fcomp = 1.27 and qobs= 1.59, n.s.).
8. Interpret your answer. The drug appears to help alleviate depression. However, as there is no
significant difference between taking a low or moderate dose, a low dose would be
recommended.

PROBLEM 3
A manufacturer of television sets is interested in the effect on tube conductivity of four different types
of coating for color picture tubes. The following conductivity data are obtained.
Coating Type Conductivity
1 143 141 150 146
2 152 149 137 143
3 134 136 132 127
4 129 127 132 129

Test the null hypothesis that H 0  1   2  3   4 , against the alternative that at least two of the
means differ. Use α = 0.05.

Solution:

Source of variation Sum ofSquares Degrees of Freedom Mean Square F Ratio

Between samples SSB= 844.68750 3 281.56250 MS B
Within samples SSW = 236.25000 12 19.68750
 14.30
MSW
Total SST = 1080.93750 15
Null Hypothesis H 0  1   2  3   4
Alternative Hypothesis H1: At least two means differ
Test Statistic: Fc = 14.30
Table Value F0.05,(3,12)= 3.49
Conclusion: Fc > F0.05,(3,12) , Reject Null Hypothesis

PROBLEM 4
A manufacturer suspects that the batches of raw material furnished by her supplier differ significantly
in calcium content. There is a large number of batches currently in the warehouse. Five of these are
randomly selected for study. A chemist makes five determinations on each batch and obtains the
following data.

Batch 1 Batch 2 Batch 3 Batch 4 Batch 5

23.46 23.59 23.51 23.28 23.29
23.48 23.46 23.64 23.40 23.46
23.56 23.42 23.46 23.37 23.37
23.39 23.49 23.52 23.46 23.32
23.40 23.50 23.49 23.39 23.38

Is there a significant variation in calcium content from batch to batch? Use

α = 0.05.

Solution:

Source of variation Sum ofSquares Degrees of Freedom Mean Square F Ratio

Between samples SSB= 0.0969760 4 0.0242440 MS B
 5.54
Within samples SSW = 0.0876000 20 0.0043800 MSW
Total SST = 0.1845760 24

H 0  1   2  3   4
H1: At least two means differ
Test Statistic: Fc = 5.54
Table Value F0.05,(4,20)= 2.84
Conclusion: Fc > F0.05,(4,20) , Reject Null Hypothesis.

PROBLEM 5

Four Laboratories measure the tin coating weight of 12 disks and that the results are as follows.

Lab A 0.25 0.27 0.22 0.30 0.27 0.28 0.32 0.24 0.31 0.26 0.21 0.28

Lab B 0.18 0.28 0.21 0.23 0.25 0.20 0.27 0.19 0.24 0.22 0.29 0.16

Lab C 0.19 0.25 0.27 0.24 0.18 0.26 0.28 0.24 0.25 0.20 0.21 0.19

Lab D 0.23 0.30 0.28 0.28 0.24 0.34 0.20 0.18 0.24 0.28 0.22 0.21

Construct an ANOVA table and test the hypothesis , whether there is any difference among the four
sample means can be attributed to chance at 5%

Solution:

Source of variation Sum ofSquares Degrees of Freedom Mean Square F Ratio

Between samples SSB= 0.013 3 0.0043 MS B
 2.87
Within samples SSW = 0.0679 44 0.0015 MSW
Total SST = 0.0809 47

H 0  1   2  3   4
H1: At least two means differ
Test Statistic: Fc = 2.87
Table Value F0.05,(3,44)= 2.82
Conclusion: Fc > F0.05,(3,44) , Reject Null Hypothesis.

PROBLEM 5

A production manager wishes to test the effect of 5 similar milling machines on the surface of finish of
small casting. So he selected 5 such machines and conducted the experiment with four replication
under each machine as per ‘Completely Randomized Design’ and obtained the following reading
Machines
 M1 M2 M3 M4 M5
25 10 40 27 15
Relication 30 20 30 20 8
16 33 49 35 45
36 42 22 48 34

Perform the required ANOVA and state the inference at 5% i.o.s.

Solution:

Source of variation Sum ofSquares Degrees of Freedom Mean Square F Ratio

Between samples SSB= 303.5 4 75.875 MS B
 0.4502
Within samples SSW = 2528.25 15 168.55 MSW
Total SST = 2831.78 19

H 0  1   2  3  4  5
H1: At least two means differ
Test Statistic: Fc = 0.4502
Table Value F0.05,(3,44)= 3.06
Conclusion: Fc > F0.05,(3,44) , Accept Null Hypothesis.
There is no significant difference between machines in terms of surface finish of small castings.

RANDOMIZED BLOCK DESIGN OR TWO-WAY ANALYSIS OF VARIANCE

Two-way (or multi-way) ANOVA is an appropriate analysis method for a study with a quantitative
outcome and two (or more) categorical explanatory variables. This is an extension of the one factor
situation to take account of second factor. As such it is often called a Blocking Factor because it places
subjects or units into homogeneous groups called Blocks. The design itself is called a Randomized
Block Design. The usual assumptions of Normality, equal variance, and independent errors apply. If an
experiment has a quantitative outcome and two categorical explanatory variables that are defined in
such a way that each experimental unit (subject) can be exposed to any combination of one level of
one explanatory variable and one level of the other explanatory variable, then the most common
analysis method is two-way ANOVA. Because there are two different explanatory variables the effects
on the outcome of a change in one variable may either not depend on the level of the other variable
(additive model) or it may depend on the level of the other variable (interaction model).

Assumptions

1. The population at each factor level combination is (approximately Normally Distributed)

2. These normal populations have a common variance, σ2.
3. The effect of one factor is the same at all levels of the other factor.
Notations

Number of levels of row factor r

Number of levels of column factor c
Total number of observations rxc
Observation in (ij) th cell of the table xij
(ith level of row factor and i = 1,2,…,r
j th level of column factor) j = 1,2,…,c
Sum of c observations in i thi row TRi   xij
j

Sum of r observations in j th column TCj   xij

i
Sum of all r x c observations T   xij   TRi   TCj
i j i j

Computational Formulae
Total Sum of Squares T2
SST   x  2
ij
i j rc
Between Rows Sum of Squares TRi2 T 2
SS R   
i c rc
Between Columns Sum of Squares TRi2 T 2
SS C   
i r rc
Error(residual) Sum of Squares SSE = SST – SSR – SSC

ANOVA TABLE

Source of variation Sum ofSquares Degrees of Freedom Mean Square F Ratio

Between rows SSR r-1 MSR MS R
Between Columns SSC c-1 MSC MS E
MS C
Error(residual) SSE (r – 1) x (c – 1) MSE
MS E
Total SST r x c -1

H0 : No effect due to row factor H0 : No effect due to column factor

H1: An effect due to row factor H1: An effect due to column factor
Critical region F > Fα,(r-1,(r-1)(c-1)) Critical region F > Fα,(c-1,(r-1)(c-1))
MS R MS C
Test Statistic FR  Test Statistic F C 
MS E MS E

PROBLEM 1
Three laboratories, A, B, and C, are used by food manufacturing companies for making nutrition
analyses of their products. The following data are the fat contents (in grams) of the same weight of
three similar types of peanut butter.

Laboratory
Peanut A B C D
Butter
Brand 1 16.6 17.7 16.0 16.3
Brand 2 16.0 15.5 15.6 15.9
Brand 3 16.4 16.3 15.9 16.2

Analyse the data at 5% significance by (a) carrying out a one-way ANOVA to see if there is a difference
between the fat content of the three brands; (b) performing a two-way ANOVA to see if there is any
difference between the Brands using the laboratories as blocks. (c) Do you think there is any evidence
that the results were not reasonably consistent between the four laboratories?
a) One-way ANOVA
Laboratory
Peanut Butter A B C D Mean
Brand 1 16.6 17.7 16.0 16.3 16.65
Brand 2 16.0 15.5 15.6 15.9 15.75
Brand 3 16.4 16.3 15.9 16.2 16.20
Mean 16.33 16.50 15.83 16.13 16.20

Sums of squares
Total SS: Inputting all the individual values into the calculator gives the following summary statistics: n
= 12, x = 16.20, sn = 0.546  nsn2 = 3.58

x
Between Brands SS: The mean scores x1 = 16.65, x 2 = 15.75 and 3 = 16.20
Each of these means came from 4 values so inputting the means with a frequency of 4 gives: n = 12, x
= 16.20, sn = 0.367  nsn2 = 1.62 (n and x for checking)

Error SS: 3.58 – 1.62 = 1.96

Anova table In this example: (k =3 brands, N =12 values)

Source S.S. d.f. M.S.S. F
Between 1.62 3-1=2 1.62/2 = 0.81 0.81/0.22 = 3.72
brands
Errors 1.96 11 - 2 = 9 1.96/9 = 0.22
Total 3.58 12 - 1 = 11
Hypothesis test
H0: 1 = 2 = 3 H1: At least two of them are different.
Critical value: F0.05 (2,9) = 4.26 (Deg. of free. from 'between brands' and 'errors'.)

Test Statistic: 3.72

Conclusion: T.S. < C.V. so H0 not rejected. There is no difference between the fat content of the
brands.

b) Two-way ANOVA

Sums of squares
From (a): Total SS: nsn2 = 3.58 Between Brands SS: nsn2 = 1.62

Between Labs Sum of Squares: Mean scores x A = 16.33, x B = 16.50, x C = 15.83, x D = 16.13
Each of these means came from 3 values so inputting the means with a frequency of 3 gives: n = 12, x
= 16.20, sn = 0.249  nsn2 = 0.75 (n and x for checking)

Error SS: 3.58 – (1.62 + 0.75) = 1.21

Anova table In this example: (k =3 brands, N =12 values)

Source S.S. d.f. M.S.S. F

Between 1.62 3-1=2 1.62/2 = 0.81 0.81/0.20 = 4.05
brands
Between 0.75 4–1=3 0.75/3 = 0.25 0.25/0.20 = 1.25
labs
Errors 1.21 11 - 5 = 6 1.21/6 = 0.20
Total 3.58 12 - 1 = 11
Hypothesis test for Brands
H0: 1 = 2 = 3 H1: At least two of them are different.

Critical value: F0.05 (2,6) = 5.14 (Deg. of free. from 'between brands' and 'errors'.)

Test Statistic: 4.05

Conclusion: T.S. < C.V. so H0 not rejected. There is no difference between the fat content of the
brands. Blocking has not changed to conclusion even though the test statistic has increased.

c) Hypothesis test for Laboratories

H0: A = 2 = C = D H1: At least two of them are different.

Critical value: F0.05 (3,6) = 4.76 (Deg. of free. from 'between brands' and 'errors'.)
Test Statistic: 1.25

Conclusion: T.S. < C.V. so H0 not rejected. The results between the different laboratories are
consistent.

PROBLEM 2

The following data represent the number of units of production per day turned out by 5 different workers
using 4 different types of machines

MACHINE TYPE
W A B C D
O 1 44 38 47 36
R
K 2 46 40 52 43
E 3 34 36 44 32
R
S 4 43 38 46 33
5 38 42 49 39

a) Test whether the five men differ with respect to mean productivity.
b) Test whether the mea productivity is same for four different machine types. Take α = 5%

Solution
We shift the origin to 40 and subtract 40 from the given values and work out with new values of xij.

MACHINE TYPE Ti Ti 2
W
O
A B C D r x
j
2
ij

R 1 4 -2 7 -4 5 6.25 85
K
2 6 0 12 3 21 110.5 189
E
R 3 -6 -4 4 -8 -14 49.0 132
S 4 3 -2 6 -7 0 0 98
5 -2 2 9 -1 16 16 90
Ti 5 -6 38 -17 T = 20 Ti 2 594
 r
=181.1

Ti 2 5 7.2 288.8 139 Ti 2

c
 c =358.8
x
i
2
ij
101 28 326 139 594
 TRi2 T 2 181.5 – 20 = 161. 5
SS R   
i c rc
TRi2 T 2 358.5 – 20 = 338. 8
SS C   
i r rc
SSE = SST – SSR – SSC 574 – (161.5 + 338.8)= 73.7

Source of S.S. d.o.f. M.S.S F

Variation
Between row 161.5 c- 1 = 4 40.375 40.374/6.142 = 6.57
Workers
Between Columns 338.8 r–1=3 11.933 112.933/6.142 = 18.39
(Machines)
Errors 73.7 12 6.142 -
Total 574 19 - -

F0.05,(4, 12) = 3.26 F0.05,(3, 12) = 3.49

F > F0.05,(4, 12) with respect to rows, hence 5 workers differ significantly.

F > F0.05,(3, 12) with respect to columns, hence 4 machine types also differ significantly in mean
productivity.
LATIN SQUARE DESIGN

A n x n LATIN Square is a square array of n distinct letters, with each appearing once and only once in
each row and in column
Example:
A B C D
B C D A
C D A B
D A B C

NOTATIONS:

Number of levels of row factor n

Number of levels of column factor n
Number of levels of treatment factor k
Sum of c observations in 15t h row TRi   xij
j

Sum of r observations in j th column TCj   xij

i
Sum of k observations in k th teatment TK   xij
k
Sum of all r x c observations T   xij   TRi   TCj
i j i j

Computational Formulae
Total Sum of Squares T2
SST   x  2 2
ij
i j n
Between Rows Sum of Squares TRi2 T 2
SS R   
i n n2
Between Columns Sum of Squares TRi2 T 2
SSC   
i n n2
Between treatment sum of squares TK2 T 2
SSTk  
i n n2
Error(residual) Sum of Squares SSE = SST – SSR – SSC - SSTk
ANOVA TABLE
Source of variation Sum ofSquares Degrees of Freedom Mean Square F Ratio
Between rows SSR n-1 MSR = SSR/(n-1) MS R
Between Columns SSC n-1 MSC = SSC/(n-1) MS E
MS C
Between SSE n-1 MSE = SSTk/(n-1)
Treatments MS E
MSTk
Error(residual) SSE (n– 1) x (n – 2) MSE = SSE/(n-1) MS E
Total SST n2 -1

FR, FC, FTk follows (n-1), (n-1)(n-2) d.o.f