LECTURE NOTES

Multiple Factor Designs

Sept 8-Sept 20
Day7-Two or More Factor Designs

RCBD
LSD
ANCOVA
Factorial Designs
Introduction
What happens when theres more than one
factor?
Vary one factor at a time
Study the factors jointly
Situations
One controllable confounding factor (RCBD)
More than one controllable confounding factor (LSD)
One or more recordable but uncontrollable factors
(ANCOVA)
Several factors of interest (Factorial Design)



?
Confounding factors are factors which we strongly
expect to have an influence on the dependent
variable (Y) but which are not the primary factor
that we wish to test for effects on Y.
A nuisance or confounding factor is a factor that
probably has some effect on the response, but it is
of no interest to the experimenterhowever, the
variability it transmits to the response needs to be
minimized
Typical nuisance factors include batches of raw
material, pieces of test equipment, time (shifts,
days, etc.), different experimental units
Physical entities with similar characteristics (plots of
land, genetically similar animals or litter mates)


These variables are not being controlled by
the analyst but can have an effect on the
outcome of the treatment being studied

If they are unknown we hope to have controlled
them via randomization
If they are known and controllable blocking
If they are known but uncontrollable - ANCOVA

Blocking is a type of constrained randomization that
can be used to control confounding by creating a "block"
or homogenous strata within which we will be able to
examine all of the treatments.

This avoids the possibility that the treatments could be
imbalanced with respect to the confounding factor(s)
resulting in a confusion as whether the results we get are
due more to an unfortunate arrangement of the
confounders than the treatment effects themselves.

RCBD- Randomized Complete Block Design
The key objective in blocking the
experimental units is
To make them as homogenous as possible
within blocks with respect to the response
variable under study
To make the different blocks as hetrogenous
as possible with respect to the response
variable under study
When each treatment is included only once
in each block, it is called RCBD

The reason for blocking is that we hope to reduce
the error sum of squares by explaining an
additional component of that error with the
variation within blocks.
If successful this results in a much smaller value for the
MSE (more precise results than CRD).
A smaller MSE will make the value of the test statistic
bigger, however a smaller number of degrees of
freedom for MSE will make the critical value of the
100(1-o) percentile point of the F distribution larger.
Generally though, the change in MSE will have a much
greater effect on the test statistic making the test more
powerful.
This variance reducing design, in addition to
testing factor of interest, it could also help with
understanding

If process is robust to nuisance conditions
Blocking is necessary in future experiments

It is highly desirable that the Experimental units
within each block are processed together
whenever this will help to reduce experimental
error variability
Example: if the experimenter might change the
administration of the experiment to the subjects over
time, consecutive processing of EUs block by block
will reduce such sources of variation from the within
blocks leading to more precise results

In setting up a randomized block experiment with
a levels of the treatment factor and b blocks, we
can have the block represent either a random or a
fixed factor.

Random blocks would correspond to a situation
where we have sampled a group of b levels from a
bigger population of possible blocking levels.
Hence the eventual conclusions we draw from the
experiment can be extrapolated to the larger population
from which we sampled.
Fixed blocks correspond to the situation where we
have chosen to examine specific set of b blocking
levels
and the results of our experiment only apply to those b
levels (no extrapolation to other levels).
Example 1:
experiment on the effects of vitamin C on the prevention of colds.
868 children randomly assigned: treatment (500mg,1000mg of
vitamin c) and a placebo (identical tablet with no vitamin C) on a
daily basis.
response of interest = number of colds contracted by each child.
The study showed No difference in average number of colds in the
treatment groups and the placebo group.
Other factors that may affect the number of colds contracted might
include, gender, age, nutritional habits of the child, etc.
These factors that may affect the response but are not of primary
interest to the investigator are referred to as nuisance or
confounding factors.

In blocked experiments the heterogeneous experimental units are
divided into homogenous subgroups called blocks and separate
experiments are conducted in each block.
For example blocking by gender would mean doing the experiment
on males and females separately.
Example 2:
An investigator is interested in testing the effects of
drugs A and B on the lymphocyte count in mice by
comparing A,B and Placebo,P.
In designing the experiment, he assumed mice from the
same litter would be more homogenous in their
response than would mice from different litters.
He arranged the experiment in an RCBD design with
three litter-mates forming each block and a total of 7
blocks.
In each block the litter mates were randomly assigned
to the treatments resulting in the following data after
conclusion of the experiment (lymphocyte count given in
units of 1000 per cubic mm of blood)
lymphocyte count in mice
Blocks
treatment 1 2 3 4 5 6 7 mean
P 5.4 4.0 7.0 5.8 3.5 7.6 5.5 5.54
A 6.0 4.8 6.9 6.4 5.5 9.0 6.8 6.49
B 5.1 3.9 6.5 5.6 3.9 7.0 5.4 5.34
mean 5.50 4.23 6.80 5.93 4.30 7.87 5.90 5.79
Analysis assuming CRD:
effects of treatment on the lymphocyte count in mice

Source DF
Sum of
Squares
Mean
Square
F
Value Pr > F
Treatment 2 5.22 2.61 1.47 0.256
Error 18 32.02 1.78
Corrected
Total
20 37.24
What will be the difference if we analyze the data
taking into account the blocking by litter effect?
The RCBD Model
y
ij i j ij
o | = + + +
i = 1,2,, a j = 1,2,,b
y
ij
= the observation in i
th
treatment in the j
th
block
= overall mean
o
i
= the effect of the i
th
treatment
|
j
= the effect of the j
th
block
c
ij
= random error
No interaction
between blocks
and treatments

Properties of the model

Sum of o
i
is zero
Sum of |
j
is zero
E(c
ij
) = 0 which implies E(Y
ij
) =
ij
=
and
Var(Y
ij
) = o
2

Y
ij
~ N(
ij
, o
2
)
Cov(c
ij
, c
ik
) = 0
Cov(c
ij
, c
lk
) = 0
i j
o | + +
j k =
and j k i l = =
0
j
|

=
1
0
a
i
i
o
=
=

The additive model implies that the expected values

of observations in different blocks for the same
treatment may differ, but the treatment effects are
the same for all blocks




There is a possibility for interaction between blocks
and treatment (Tukeys additivity test)



( )
ij i j
E Y o | = + +

Statistical Inference

Under the stated assumptions, we could use
OLS or MLE to estimate parameters

Hypothesis




Partition sum of squares - Two way ANOVA

1 2
0 :
1 : 0
a
H
H Not H
= = =
SST
(total sum of squares)
SStr
(treatment
sum of squares)
SSE
(error sum of squares)
SSB
(sum of squares
blocks)
SSE
(sum of squares
error)
TWO WAY ANOVA
Individual
observations
.
.
.
.
.
.
.
.
.
.
.
.
Single Independent Variable
Blocking
Variable
.
.
.
.
.
Randomized Block Design
partition the total sum of squares (SST) ,
in to three components (SStr, SSb and
SSE)






2 2
2 2
.. . .. . .. . . ..
1 1
( ) ( ) ( ) ( )
a b a b a b
ij i j ij i j
i j i j i j
SStr
SST SSb SSE
Y Y b Y Y a Y Y Y Y Y Y
= =

= + + +
TWO way ANOVA for RCBD
The degrees of freedom for the sums of squares in



are as follows:


Ratios of sums of squares to their degrees of
freedom result in mean squares, and
We could use Cochrans theorem to decide about the
distribution of the ratio of the mean squares
used to test the hypothesis H0:equal treatment
means
SST SStr SSB SSE = + +
1 ( 1) ( 1) [( 1)( 1)] ab a b a b = + +
Expected mean squares
E(MSE) =o
2
E(MStr) = o
2
+



Exercise: 95% CI for o
2

Thus, we could test the treatment effect
hypothesis H0: all o
i
=0 vs H1: not H0 by the
statistic


2 2
2
( )
1 1
a a
i i
i i
b b
a a
o
o

= +

2
2
( )
1
j
j
a
E MSB
b
|
o

= +

~ ( 1, ( 1)( 1); 0)
MStr
F F a a b
MSE
=
Under the specific alternative hypothesis with given
values for the o
i
's, this test statistic has a non-central
F distribution with non-centrality parameter given by
lambda,



Where


~ ( 1, ( 1)( 1); )
MStr
F F a a b
MSE
=
2
2
, which is same as in the CRD
a
i
i
b o

=
ANOVA Table
Source of
variation
Degrees of
freedom
a

Sums of
squares (SSQ)
Mean
square (MS)
F
Blocks (B) b-1 SSB SSB/(b-1) MSB/MSE
Treatments (Tr) a-1 SStr SStr/(a-1) MStr/MSE
Error (E) (a-1)*(b-1) SSE SSE/((a-1)*(b-1))
Total (Tot) a*b-1 SST
a
where a=number of treatments and b=number of blocks or replications.

Exercise:
If only two treatments are investigated
(a=2) in RCBD, it can be shown that the F
test for treatment effects given above is
equivalent to the two sided t-test for
paired observations
Blocking effect
the test will be more powerful here for the same values of b (r in
the previous case) and the o
i
's, because if the blocking was
appropriate (i.e. if that factor had a pronounced effect on outcome)
we will usually have a much smaller error variance than if we did
not block, resulting in a much bigger non-centrality value.
However, we will be looking at power under a slightly different
condition of having a smaller df for MSE.
While this will require a larger critical value of significance
(reducing the power if all other things were held equal), this is
usually more than made up for by the large reduction in error
variance o
2
achieved by the design.
Beware that if your blocks had no really important effect on
outcome, you could potentially lose power by blocking.
Thus, it is important to block only when you have solid evidence
that you are likely to gain something by adding this feature.
Blocking effect
Successful blocking minimizes variance
among units within blocks while
maximizing the variance among blocks.

Since, precision usually decreases as the
number of experimental units per block
increases, block size should be kept as
small as possible.
Analysis under RCBD: testing the effects of drugs
A and B on the lymphocyte count in mice
Source DF
Sum of
Squares
Mean
Square
F
Value Pr > F
Treatment 2 5.22 2.61 17.93 0.00005
Blocking 6 30.28 5.05 34.71 <0.00001
Error 12 1.74 0.15
Corrected
Total
20 37.24
Do we need to test block effect?
Usually not of interest (blocked for a reason)
Blocks are not randomized to experimental units
Can compute ratio of variation explained by blocking to understand
the impact of blocking
Trade-off: reduction in variance vs loss in degrees of freedom
Relative efficiency







Ultimately, the loss in df will have little effect as long as a moderate
number of error degrees of freedom are available


2
:
2
2
( 1)( 3)
( 3)( 1)
, are error variances
( 1)( 1)
( 1)
RCBD CRD CRD
RCBD CRD
RCBD CRD RCBD
RCBD
CRD
v v
RE
v v
where
v a b
v a b
o
o
o
+ +
=
+ +
=
=
2
( 1) ( 1)

1
crd
b MSB b a MSE
ab
o
+
=

Please change t by a.
Assumptions
Normality
Histogram and probability plot (qqplot)
Additivity
Tukeys test of additivity (block x trt interaction)
If significant, it means block effect different for
different treatments
Log transformation could eliminate interaction (non-
addititvity)
eg. E(y
ij
)=o
i
|
j
then log(yij)= + o
I
+ |
j
+e
ij
Constant variance (check by treatment and block)
He introduced the word "bit" as a
contraction of binary digit.

He used the term "software" in a
computing context in a 1958 article

And also pioneered many statistical
methods

articulated the important distinction
between exploratory data analysis and

He retired in 1985. In 2000, he died in
New Brunswick, New Jersey.

?
Checking Additive assumption
(four approaches)
Tukeys test of additivity (block x trt interaction)
Plot of residuals against fitted values
A curvilinear pattern of the residuals suggests the
presence of interaction and also suggests non-
constancy of variances
A more effective plot is plot of the responses Y
ij

by blocks
X-axis is treatment, y-axis is response and the
overlayed lines are for each block.
Lack of parallelism is strong indication that blocks and
treatment interact in their effects on the response

Interaction test with a single replication per cell

Having a single replication per cell has so far
prevented us from testing for an interaction effect
similar to what is done in factorial designs.
But, there is a special type of interaction that we
could test, called Tukeys one degrees of freedom
test of non-additivity.



We are interested to know if this model is any
better than just the simple additive model?

, 1, .., ; 1, ..,
ij i j i j ij
Y i a j b o | o | c = + + + + = =


If the
i
and
j
were known, we might use a
least squares approach to obtain an
estimate of lambda.


That is, find estimate of lambda such that
minimizes the above expression,
2
1 1
[ ( )]
a b
ij i j i j
i j
Y o | o |
= =
+ + +
, 1, .., ; 1, ..,
ij i j i j ij
Y i a j b o | o | c = + + + + = =
Taking the usual estimates of the
parameters ,
i
and
j
, and minimizing this



we get,


2
1 1
[ ( )]
a b
ij i j i j
i j
Y o | o |
= =
+ + +
1 1
2 2
1 1

a b
i j ij
i j
a b
i j
i j
Y o |

o |
= =
= =


=
Let
d
ij
= and since

then we can think of this as a contrast in the means for Y
ij
with
one replication per cell.
Re-writing the above expression after proper substitution
we get Tukeys sum of squares for non-additivity,




And, since this is a contrast, it will have one degrees of
freedom.

0
a b
i j
i j
o | =
2
1 1
2
1 1
{ }
a b
ij ij
i j
nonadd a b
ij
i j
d Y
SS
d
= =
= =


=

i j
o |
Let,
- SSremainder= SSE SSnonadd
- (a-1)(b-1)-1 = (ab a b+1)-1=ab-a-b.

SSnonadd and SSremainder are orthogonal to each other
and hence are statistically independent (Cochrans Thm).

Thus we can test

H0: = 0 vs H1: not H0



with

~ (1, , 0)
/ ( )
nonadd
nonadd
remainder
SS
F F ab a b
SS ab a b
=


Q. What do we do if we accept H0 or do not reject H0?
A. most statisticians would pool SSnonass and SSremainder into SSE
and do the usual tests for the main effect of treatment and blocking
In general in order to reduce type II error rate, a liberal type I error
rate is used for interaction test (alpha>10%)
Q. what do we do if we reject H0?
A1. if the true model structure was not additive and went ahead and did
the usual main effects test using F=MStr/MSE, then
we will have a type-I error level that is smaller than the nominal.
We would get too few significant results and the testing procedure would
be conservative.
However, if we get a significant result with this test we might
feel that it would also have been significant with the proper kind
of test based on a non-additive model.

A2. Make efforts to remove it via transformations of Y (eg. Sqrt, log)

Day 8-Regression Approach to RCBD
Example: consider RCBD with 3 blocks and 2 levels of Treatment B.
Block as fixed effect and model of the form
y
ij
= + o
i
+|
j
+ e
ij









y =
X =
1
2
3
1
2
B

|
|
|
o
o
(
(
(
(
=
(
(
(
(

Regression approach to test additivity
1. Fit additive model
2. Obtain residuals, r
ij

3. Fit additive model


4. Obtain residuals from (3) r
ij

5. Tukey sum of squares is

6. F=TSS/MSE ~F(1,ab-a-b,0)
2
..

2
ij
ij i j ij
y
y
y
o | c = + + + +
ij i j ij
y o | c = + + +
2 2
'
ij ij
TSS r r

=
Multiple Comparison in RCBD
Similar to procedures in CRD
g(n) is replaced by a(b) in formula
Degree of freedom for error is (b-1)(a-1)

1 / 2;( 1)( 1)
1 ; ,( 1)( 1)
:
:
: ( 1) (1 ; 1, ( 1)( 1)
a b
a a b
t test t
Tukey q
Scheffe a F a a b
o
o
o

There are some variations to simple
RCBD.
RCBD with replicates within blocks
Incomplete block designs (block designs with
fewer EUs per block than treatments)
More than two directional blocking (LSD,
GLSD,)
Variations to simple RCBD
An experiment was designed to study the
performance of four different detergents for
cleaning clothes.
The following cleanness readings (higher =
cleaner) were obtained with specially designed
equipment for three different types of common
stains (blocking factor).
Is there a difference among the detergents?;
Example: Deteregent Study
Replicated RCBD
Advantages of replicated RCBD:
The natural block size may result in more units per
block than there are treatments thus allowing for
within block replication
Within block replication allows for the separation of
block*treatment interaction from experimental error,
which may improve the interpretation of results when
the block*treatment interaction is significant
The within block replication may be used to assign
extra replication to selected treatments to increase
sensitivity for comparisons of interest
**with the key disadvantage that a large block
size (if it is not the natural block size) reduces
the effectiveness of the blocking
It is also called Generalized RCBD
In replicated RCBD the model is,

Y
ijk
= +
i
+
j
+ ()
ij
+ c
ijk
,
i=1,,a;
j=1,,b;
k=1,,s

where
Y
ijk
is the response for the kth subject in j-th
block and i-th group ,
(o|)
ij
is the interaction effect of the ith
treatment with jth block
ANOVA Table
Source of
variation
Degrees of
freedom
a

Sums of
squares (SS)
Mean
square (MS)
F
Blocks (B) b-1 SSB SSB/(b-1) MS
B
/MS
E

Treatments (Tr) a-1 SStr SStr/(a-1) MS
Tr
/MS
E

Block*Treatment (B*T) (a-1)*(b-1) SSBT SSBT/(a-1)*(b-1) MSBT/MSE
Experimental Error (E) a*b(s-1) SSE SSE/a*b(s-1)
Total (Tot) a*b*s-1 SST
a
where a=number of treatments, b=number of blocks and s=number of replications.
Expected Mean Squares
Both treatment and block are fixed


E(MSE) =o
2
E(MStr) = o
2
+ sb

E(MSb) = o
2
+ sa

E(MStb) = o
2
+ s

2
1
a
i
i
a
o

2
1
b
j
j
b
|

2
1
( )
( 1)( 1)
b a
ij
j i
a b
o|
=


1
0 : ... 0
/
a
H
F Mstr MSE
o o = = =
=
Expected Mean Squares
Only blocks are random and hence
interaction too is random

E(MSE) =o
2
E(MStr) = o
2
+ so
2
tb
+sb
E(MSb) = o
2
+ sao
2
b

E(MStb) = o
2
+ so
2
tb
2
1
a
i
i
a
o

1
0 : ... 0
/
a
H
F Mstr MStb
o o = = =
=
Expected Mean Squares
Both effects are random

E(MSE) =o
2
E(MStr) = o
2
+ sbo
2
t
+ so
2
tb

E(MSb) = o
2
+ sao
2
t
+ so
2
tb

E(MStb) = o
2
+ so
2
tb
2
0 : 0
/
H
F Mstr MStb
o
o =
=

RCBD (one replication) with random block effect

If the blocks are random effects then
|
j
~N(0 , o
b
2
)
E(c
ij
) = 0 which implies E(Y
ij
) =
i
=
and
Var(Y
ij
) = o
2
+ o
b
2

Y
ij
~ N(
i
, o
2
+ o
b
2
)
Cov(c
ij
, c
ik
) = o
b
2
( )
Cov(c
ij
, c
lk
) = o
b
2
( )
i
o +
j k =
and j k i l = =
Expected Mean Squares
For RCBD with single replication, if only
blocks are random

E(MSE) =o
2
E(MStr) = o
2
+ b
E(MSB) = o
2
+ ao
2
b


2
1
a
i
i
a
o

1
0 : ... 0
/
a
H
F Mstr MSE
o o = = =
=
- Think of an unbiased estimator for o
2
b

Incomplete Block Design (IBD)
We will see the analysis methods for IBD later
IBD is an RCBD design in which there are fewer
experimental units per block than treatments.
One type of this design is balanced incomplete block
design (BIBD).
In this design every treatment pair occurs within a block
exactly the same number of times.
The reason for this type of design is to take advantage of
greater efficiency of smaller block sizes.
Example: suppose we are interested in testing four
mosquito repellents.
The natural block size is two (our two arms), but we have 4 trts.
The following design is proposed where the data are numbers of
mosquito bites during a specified period of time.
BIBD design example
Treatments
Subject A B C D
1 12 9
2 9 7
3 18 11
4 3 4
5 5 9
6 13 10
Treatment
mean
13 9 8 10
- It is incomplete because not all treatments occur in each block
- it is considered balanced in the sense that each pair of treatments occurs together
(with blocks) exactly the same number of times.
Example 1: Insurance premium example

An analyst in insurance company A studied
the premium for auto insurance in six cities.
The six cities were selected to represent
different regions (East, West) and different
sizes (small, medium and large).
Response= three months premium charges
for a certain category of risk.
The interest is to study the effect of city size
controlling for geographical region.
ANOVA
Source df SS MS F Pvalue
city 2 9300 4650 93 0.0106
Region 1 1350 1350 27 0.0351
Error 2 100 50


region
city E W ave
small 140 100 120
med 210 180 195
large 220 200 210
ave 190 160 175
Data Example: Insurance premium example
data insurance;
input premium city
region;
datalines;
140 1 1
100 1 2
210 2 1
180 2 2
220 3 1
200 3 2
;
proc glm data=insurance;
class city region;
model premium = city
region;
means city region
/tukey;
run;
quit;

ANOVA
Source df SS MS F Pvalue
city 2 9300 4650 93 .
Region 1 1350 1350 27 .
City*Reg 2 100 50
Error 0 . .




Tukeys test
Obs msa msb ssab ssrem f p_value
1 4650 450 87.0968 12.9032 6.75 0.23391


Creating RCBD in SAS
A.
B. use randomized block design and randomize the four
treatments to four flowers within each type


Text Book Example
=number of lever presses/elapsed time of the session
Trt= 5 dosages of drug in mg/kg
(Page 121 of JL)
What is the EU?
Two Way ANOVA
Examining Trend: since factor is quantitative
2
( 1) ( 1)( 1)
( 1)
9(0.185) 9(5 1)(0.0083)
0.0408
5 (10 1)
crd
b MSB a b MSE
a b
o
+
=

+
= =

2 2
2
crd rcb
crd
o o
o

=
Summary
Non-replicated RCBD:
When experimental units represent physical entities
Smaller blocks of EUs usually result in greater homogeneity
The larger the blocks, the less homogenous

Replicated RCBD
When EUs represent trials rather than physical
entities and the experimental runs can be made
quickly,
larger block sizes may not increase variability of EUs with in
a block
Summary
Advantages of replicated RCBD
More error degrees of freedom
Interaction and error are not confounded
Can separate error and interaction SS
Easier assessment of additivity
Is good if blocks are expensive but
observations are cheap
Consider example: tee height (golf)
Example of Generalized RCBD
Page 128 of text book
Objective

To determine if tee height affects golf driving distance
(a)purpose

To recommend what
tee height to use
(b) Identify sources of
variation

- tee height
- Golfer and ability level
- brand ball
- club
- wind speed
- repeat swings
c) Choose rule to assign experimental units to treatment
factors

Complete Block Design: randomize the order that each golfer
Hit a ball from each of the tee height

- Blocks will be Golfers (takes into account differences
in ability levels and clubs)
random sample of golfers? (9 golfers)

- Treatment Factor tee height,
each golfer will hit 5 balls from each tee height
in a randomized order
d) Measurements to be made:
1) distance
Note:
-In the middle table pvalue<0.0001 (incorrect since uses MSE as denominator
- pvalue in bottom table is correct
-If interaction is not significant (o>0.1)- redo analysis without interaction
-Since treatment effect is significant we could investigate further on pairwise
-Note that the error term is block*trt
-Conclusion: tee your golf ball up so that half of the ball is above the crown
of the driver club-face to maximize distance
The power of the F test for treatment effects for RCBD involves the same
non-centrality parameter as for CRD
But, the two lead to different power levels. Why?
variance (
2
) will differ for the two designs
degrees of freedom associated with denominator also differ

Example:
Consider the text book example for d-ampthamine
Day9: Latin Square Design (LSD)
-Due to Fisher (1935)
- agricultural experiments (eg fertility gradient of plots)
-Industrial experiments (eg. Wear life of auto tires)
-Pharmaceutical (eg. Bioequivalence study)
The Latin Square Design
This design is used to simultaneously control (or eliminate) two
independent sources of nuisance variability
It is called Latin because we usually specify the treatment by the
Latin letters
Square because it always has the same number of levels (t) for the
row and column nuisance factors
A significant assumption is that the three factors (treatments and two
nuisance factors) do not interact
More restrictive than the RCBD
Each treatment appears once and only once in each row and column
If you can block on two sources of variation (rows x columns) you
can reduce experimental error when compared to the RCBD
It further reduces variability increasing
Sensitivity to detect treatment effect

A
B C D
A
B C D A
B C D
A
B C D
In LSD every treatment occurs in every row and column
Also every row occurs in every column and vise versa
Advantages and Disadvantages
Advantage:
Allows the experimenter to control two sources of
variation
Disadvantages:
Error degree of freedom ([t-1]x[t-2]) is small if there
are only a few treatments
The experiment becomes very large if the number of
treatments is large
The statistical analysis is complicated by missing
blocks and mis-assigned treatments
The LSD Model
( ) ( )
k i j
ij k ij k
y t c = + + + +
i = 1,2,, t j = 1,2,, t
y
ij(k)
= the observation in i
th
row and the j
th
column
receiving the k
th
treatment
= overall mean
t
k
= the effect of the i
th
treatment

i
= the effect of the i
th
row
c
ij(k)
= random error
k = 1,2,, t

j
= the effect of the j
th
column
No interaction
between rows,
columns and
treatments
A Latin Square experiment is assumed to be
a three-factor experiment.
The factors are rows, columns and
treatments.
It is assumed that there is no interaction
between rows, columns and treatments.
We can partition the sum of squares into
four components
SST=SSR+SSC+SStr+SSE
Usual F test under H0 using Cochrans
theorem
The ANOVA Table for a Latin Square Experiment
Source S.S. d.f. M.S. F
p-
value
Treat SStr

t-1 MStr

MStr

/MS
E
Rows SS
Row
t-1 MS
Row
MS
Row
/MS
E

Cols SS
Col
t-1 MS
Col
MS
Col
/MS
E

Error SS
E
(t-1)(t-2)
MS
E
Total SS
T
t
2
- 1
LSD Text book Example
Purpose: to test the bioequivalence of three formulations
(A=solution, B=tablet, C=capsule) of a drug
Response: concentration of the drug in the blood as a function of
time since dosing
Three volunteers took drug in succession after washout period
After dosing, blood samples taken every hour for four hours
Since there may be variation from subject to subject metabolism,
subject is row factor
Since metabolism also could vary from time to time, time is column
The Graeco-Latin Square Design
This design is used to simultaneously control (or
eliminate) three sources of nuisance variability
It is called Graeco-Latin because we usually
specify the third nuisance factor, represented by
the Greek letters, orthogonal to the Latin letters
A significant assumption is that the four factors
(treatments, nuisance factors) do not interact
If this assumption is violated, as with the Latin
square design, it will not produce valid results

GRAECO LATIN Square Design
A Greaco-Latin square consists of two latin squares
(one using the letters A, B, C, the other using greek
letters a, b, c, ) such that when the two latin square
are supper imposed on each other the letters of one
square appear once and only once with the letters of
the other square. The two Latin squares are called
mutually orthogonal.
Example: a 7 x 7 Greaco-Latin Square
Ao Bc C| D| E_ F Go
B| C| D_ E Fo Go Ac
C_ D Eo Fo Gc A| B|
Do Eo Fc G| A| B_ C
Ec F| G| A_ B Co Do
F| G_ A Bo Co Dc E|
G Ao Bo Cc D| E| F_

The GLSD Model
( ) ( )
k l i j
ij kl ij kl
y t c = + + + + +
i = 1,2,, t
j = 1,2,, t
y
ij(kl)
= the observation in i
th
row and the j
th
column
receiving the k
th
Latin treatment and the l
th
Greek
treatment
k = 1,2,, t
l = 1,2,, t
= overall mean
t
k
= the effect of the k
th
Latin treatment

i
= the effect of the i
th
row
c
ij(k)
= random error

j
= the effect of the j
th
column
No interaction between rows, columns,
Latin treatments and Greek treatments

l
= the effect of the l
th
Greek treatment
A Greaco-Latin Square experiment is
assumed to be a four-factor experiment.
The factors are rows, columns, Latin
treatments and Greek treatments.
It is assumed that there is no interaction
between rows, columns, Latin treatments
and Greek treatments.

Analysis of Covariance
ANCOVA
Introduction
Consider factor x which is correlated with y
BUT NOT with treatment
Can measure x but can't control/predict it
(as with blocks)
Nuisance factor x called a covariate
ANCOVA adjusts y for effect of covariate x
(retrospective adjustment for bias)
Without adjustment, effects of x may
inflate
2
alter treatment comparison
Introduction
ANCOVA combines regression and ANOVA
Response variable is continuous
One or more explanatory factors (the treatments)
One or more continuous explanatory variables

The goal of ANCOVA is to reduce the error variance. This
increases the power of tests and narrows the confidence
intervals.

Analysis of covariance adjusts for measurable variables
that affect the response but have nothing to do with the
factors (treatments) in the experiment.
Model Description
Consider single covariate in CRD
Constant slope model is


Assumptions
x
ij
not affected by treatment
x and y are linearly related
Constant slope
Errors are normally and independently distributed
Equality of error variance for different trts

ij i ij ij
y x t | c = + + +
Model Description
Non-constant slope model is


Additional assumptions
x
ij
not affected by treatment
x and y are linearly related
There is interaction between x and treatment
and hence non constant slope

( )
ij i ij i ij ij
y x x t | t| c = + + + +
Examples
Pretest/Posttest score analysis: The gain in score y
may be associated with the pretest score x. Analysis of
covariance provides a way to control for pre-test
differences. That way, one does not need a group of
students with similar pretest scores and randomly
assign them to a control and treatment group.
Weight gain experiments in animals: If wishing to
compare different feeds, the weight gain y may be
associated with the original weight of the animal.
Comparing competing drug products: The effect of
the drug A after two hours (measured on a scale from
1 to 10) may be associated with the initial state of the
subject. Variables describing the initial state may be
used as covariates.
Properties of ANCOVA Model
While in ANOVA, E(Yij)=
i
, in ANCOVA this is not true
because of depends on X
ij


Mean differences are the same at any value of x


Constancy of slopes: this is a crucial assumption since
the difference between means can not be summarized
by a single number on the main effects, if violated
If treatments interact with x, resulting in non-parallel
lines, ANCOVA is not appropriate. In this case, separate
treatment regression lines need to be estimated and
then compared.

( )
ij i ij ij
E y x t | = + + =
1 2 1 2
t t =
General Approach to ANCOVA
First look at the effect of x
ij
. If it isnt significant,
do an ANOVA and be done with it.
Check to see that x
ij
is not significantly affected
by the factor values.
Test to see that | is not significantly different for
all factor levels.
This is an interaction between the factors and
the covariates.
If there is an interaction STOP!
If both tests pass, do the ANCOVA.
Model estimates
Centering of X by its mean



..
. .
2
.
. .. .

( )( )

( )

ij i ij i
ij i
i i i
y
y y x x
x x
y y x

|
t |


=

=

=
..
( )
ij i ij ij
y x x t | c = + + +
Inference
H0: t
1
=t
2
==t
g
=0
Compare treatment means after adjusting for
differences among treatments due to
differences in covariate levels.
We are not interested in testing whether
covariate (x) is significant or not



We could compute efficiency of modeling x
( | ) / ( 1)
/ ( 1)
SS trt x g
F
SSE N g

=

ANCOVA Example
Example: Data in the following example are selected
from a larger experiment on the use of drugs in the
treatment of leprosy (Snedecor and Cochran; 1967,
p. 422).
Variables in the study are as follows:
Drug: two antibiotics (A and D) and a control (F)
PreTreatment: a pretreatment score of leprosy bacilli
PostTreatment a posttreatment score of leprosy bacilli
Ten patients are selected for each treatment (Drug), and
six sites on each patient are measured for leprosy bacilli.
The covariate (a pretreatment score) is included in the
model for increased precision in determining the effect of
drug treatments on the posttreatment count of bacilli.

ANCOVA Example
data DrugTest;
input Drug $ PreTreatment PostTreatment @@;
datalines;
A 11 6 A 8 0 A 5 2 A 14 8 A 19 11
A 6 4 A 10 13 A 6 1 A 11 8 A 3 0
D 6 0 D 6 2 D 7 3 D 8 1 D 18 18
D 8 4 D 19 14 D 8 9 D 5 1 D 15 9
F 16 13 F 13 10 F 11 18 F 9 5 F 21 23
F 16 12 F 12 5 F 12 16 F 7 1 F 12 20 ;

ANCOVA Example
perform ANOVA and compute Drug LS-
means

proc glm data=DrugTest;
class Drug;
model PostTreatment = Drug / solution;
lsmeans Drug / stderr pdiff cov out=adjmeans;
run;
proc print data=adjmeans; run;
ANCOVA Example
perform a parallel-slopes analysis of covariance
with PROC GLM, and compute Drug LS-means

proc glm data=DrugTest;
class Drug;
model PostTreatment = Drug PreTreatment / solution;
lsmeans Drug / stderr pdiff cov out=adjmeans; run;
proc print data=adjmeans; run;

This model assumes that the slopes relating
posttreatment scores to pretreatment scores are
parallel for all drugs.
You can check this assumption by including the
interaction, Drug*PreTreatment
ANCOVA Example
The new graphical features of PROC GLM enable you to visualize
the fitted analysis of covariance model.

ods graphics on;
proc glm data=DrugTest plot=meanplot(cl);
class Drug;
model PostTreatment = Drug PreTreatment;
lsmeans Drug / pdiff;
run;
ods graphics off;

the SAS statements PLOTS=MEANPLOT(CL) option add
confidence limits for the individual LS-means.
If you also specify the PDIFF option in the LSMEANS statement, the
output also includes a plot appropriate for the type of LS-mean
differences computed. In this case, the default is to compare all LS-
means with each other pairwise, so the plot is a "diffogram" or
"mean-mean scatter plot" (Hsu 1996),
ANCOVA Example
ANCOVA Example
Summary of graphs
The analysis of covariance plot, Fig 1
Shows that the control (drug F) has higher
posttreatment scores across the range of
pretreatment scores,
while the fitted models for the two antibiotics (drugs A
and D) nearly coincide.
Similarly, while the diffogram, Fig 2 indicates
none of the LS-mean differences are significant,
the difference between the LS-means for the two
antibiotics is much closer to zero than the differences
between either one and the control.
Plot 1
Plot 2
Example2: with interaction
This model assumes that the slopes relating posttreatment
scores to pretreatment scores are parallel for all drugs.

The Type I SS for Drug (293.6) gives the between-drug
sums of squares that are obtained for the analysis-of-
variance model PostTreatment=Drug.
This measures the difference between arithmetic means of
posttreatment scores for different drugs, disregarding the
covariate.
The Type III SS for Drug (68.5537) gives the Drug sum of
squares adjusted for the covariate.
This measures the differences between Drug LS-means,
controlling for the covariate.
The Type I test is highly significant (p=0.001), but the Type
III test is not. This indicates that, while there is a
statistically significant difference between the arithmetic
drug means, this difference is reduced to below the level of
background noise when you take the pretreatment scores
into account.
From the table of parameter estimates, you can derive the least-
squares predictive formula model for estimating posttreatment score
based on pretreatment score and drug.






The above results show the LS-means, which are, in a sense, the
means adjusted for the covariate.
The STDERR option in the LSMEANS statement causes the standard
error of the LS-means and the probability of getting a larger t value
under the hypothesis: H0: LS-mean = 0 to be included in this table as
well.
Specifying the PDIFF option causes all probability values for the
hypothesis: H0: LS-mean(i) = LS-mean(j) to be displayed, where the
indexes i and j are numbered treatment levels.

SAS applications
Run 1: constant slopes
PROC GLM;
CLASS TRT;
MODEL Y=TRT X;
LSMEANS TRT/DIFF;
RUN;

Run 2: separate slopes
PROC GLM;
CLASS TRT;
MODEL Y=TRT X X*TRT/NOINT SOLUTION;
RUN;
Run 3: separate slopes (inflates TRT sum of squares-
order matters)
PROC GLM;
CLASS TRT;
MODEL Y=X TRT X*TRT/NOINT SOLUTION;
RUN;

Consider the previous example
Run 4: separate slopes
Test for equal slopes: Ho: all |
i
equal (|)

PROC GLM;
CLASS TRT;
MODEL Y=TRT X X*TRT/NOINT SOLUTION;
CONTRAST 'EQUAL SLOPES' X*TRT 1 0 0 -1,
X*TRT 0 1 0 -1,
X*TRT 0 0 1 -1;
RUN;
Run 5: equal slopes model:
y
ij
= + t
i
+ | x
ij
+ c
ij


PROC GLM;
CLASS TRT;
MODEL Y=TRT X/SOLUTION;
*LSMEANS TRT/DIFF;
*LSMEANS TRT/AT X=0;
ESTIMATE 'INTCPT T=1' INTERCEPT 1 TRT 1 0 0 0;
ESTIMATE 'INTCPT T=2' INTERCEPT 1 TRT 0 1 0 0;
ESTIMATE 'INTCPT T=3' INTERCEPT 1 TRT 0 0 1 0;
ESTIMATE 'INTCPT T=C' INTERCEPT 1 TRT 0 0 0 1;
ESTIMATE 'MEAN AT T=1' INTERCEPT 1 TRT 1 0 0 0 X 346.75;
ESTIMATE 'MEAN AT T=2' INTERCEPT 1 TRT 0 1 0 0 X 371.375;
ESTIMATE 'MEAN AT T=3' INTERCEPT 1 TRT 0 0 1 0 X 380.375;
ESTIMATE 'MEAN AT T=C' INTERCEPT 1 TRT 0 0 0 1 X 414.125;
RUN;

Run 1: ANOVA without Covariate
Run 2: ANOVA with Covariate, equal slopes
Run 3: ANOVA with Covariate, separate slopes
Note
The total variation in the response (SST) is equal to
the sum of the:
Variation explained by the treatment (SSA), plus the
Variation explained by the covariate, plus the
Variation explained by the interaction between the factor
levels and the covariate (hopefully small), plus the
Variation explained by the error term.
Since the factor levels and the covariate are
dependent in non-orthogonal data, fitting the
covariate first inflates the variation explained by the
treatment, potentially producing an invalid positive
result.
So put the treatment variable first in the model.
ANCOVA
Can incorporate covariate into any model
For example: constant slope model for a two-
factor model



Assume constant slope for each (i j)
combination
Can include interaction terms to vary slope
Plot y vs x for each combination
( )
ijk i j ij ijk ijk
y x o | o| c = + + + + +
Summary
If you have covariates, use them. They will
improve your confidence intervals or identify that
you have a problem.
Order matters in fitting.
In ANCOVA, fit the treatment variable first.
Youre interested in the effect of the treatment,
not of the control variable.
If the interaction between the treatment and
control variables is significant, stop!
It means the slopes differ significantly, which is a
(nasty) problem.
Summary
Effectiveness of ANCOVA can be measured as



ANCOVA and ANOVA need not necessarily lead
to the same conclusion on treatment effect
If X is pre-treatment measure of Y and If the
slope for Y on x regression is known to be one
then we could do ANOVA on Y-X instead of
ANCOVA
ANOVA ANCOVA
ANOVA
MSE MSE
RE
MSE

=
Summary
Unequal ns or unbalanced Designs
under the MCAR (Missing data completely at
random) assumption:
SAS Type III Sum of Squares provides a test of the
partial effects,
all submodels are compared to the overall model
0
ij i ij ij
i i ij
Y x
x
t | c
| | c
= + + +
= + +
Summary
SAS Type I SS
SAS model statement: (testing the equality of
slopes assumption in ancova)
model y= trt cov trt*cov;
SS(trt | )
SS(cov | , trt)
SS(trt*cov | , trt, cov)
For Type I SS, the sum of all effects add up to
the model SS:
SS(trt)+SS(cov)+SS(trt*cov)+SS(error)=SS(total)
SSs are also independent
Summary
SAS Type II SS
SAS model statement: (testing the equality of
slopes assumption in ancova)
model y= trt cov trt*cov;
SS(trt | ,cov)
SS(cov | , trt)
SS(trt*cov | , trt, cov)
For Type II SS do NOT necessarily add upto
model SS:
SS(trt)+SS(cov)+SS(trt*cov)+SS(error)SS(total)
SSs are NOT independent
Summary
SAS Type III: Partial Sum of Squares
SAS model statement: (testing the equality of
slopes assumption in ancova)
model y= trt cov trt*cov;
SS(trt | ,cov, trt*cov)
SS(cov | , trt, trt*cov)
SS(trt*cov | , trt, cov)
For Type III SS do NOT necessarily add upto
model SS:
SS(trt)+SS(cov)+SS(trt*cov)+SS(error)SS(total)
SSs are NOT independent