AS3G / STM50

1

AS3G / STM50
Study Design


Lecture 1
Introduction to Design of Experiments











AS3G / STM50
2

Contents

The Scientific Process

Definitions and Introductions to Design Terminology
o What is an experiment?
o Purpose of experiment
o Role of the experimenter
o Role of the statistician
o Objectives
o Experimental units
o Treatments
o Randomisation
o Replication
o Blocking

The Story of an experiment
AS3G / STM50
3
The Scientific Process



Observe
Reflect Generalise
Plan
Draw from your
experiences or your data
What does it say (data
analysis)?
What does it mean?
What are the possible
reasons?
Is there a general rule
or principle, which we
can exploit later?
Can we collect new data
to test the proposed rule?
AS3G / STM50
4
Design comes into all aspects of the scientific process:

Plan This is the most obvious part of the process that design influences.
Planning what data you are going to collect and where, under what conditions
etc requires some form of experimental design.

Observe Carry out the experiment according to the design and observe the
results.

Reflect The analysis and inferences you can draw from your data depend on
the data you have chosen to collect along with the design of the experiment
(example on next slide).

Generalise You can only generalise your results to people/areas/cohorts etc
that you have chosen to investigate in your design.

AS3G / STM50
5
? Analysis or Design ?

Mead (1988) page 5:

Consider an experiment to compare 2 drugs, A and B. A is well known, B is not.
There are 40 patients available.

Design 1: Allocate 1 patient to A, and 39 patients to B.
scJ
1
= o [
1
1
+
1
39

1
2
= 1. 13o

Design 2: Allocate 20 patients to A, and 20 patients to B.
scJ
2
= o [
1
20
+
1
20

1
2
= . 31o

Take home message: Design and analysis are linked!
AS3G / STM50
6
Definitions & Introductions to Design Terminology


What is an experiment?

An experiment is the conduct of one or more test procedures under controlled
conditions, so that we can study the effects of changing the conditions.

A designed experiment consists of:

1. The design (before the experiment); and

2. The analysis (after the experiment).

In this course, the analysis is assumed be carried out using Linear Models.



AS3G / STM50
7
Purpose of design

Why do we need to design experiments?
To ensure that the experiment will be adequate to answer the questions
posed.
To minimise unexplained variation.
To avoid collecting unnecessary data.
To ensure that the experiment is feasible, economical and ethical.
To ensure that the data can be analysed meaningfully.

AS3G / STM50
8
Role of the Experimenter
Formulate sensible questions to be answered by the experiment (hypotheses
to be tested).
Specify what treatments are to be given.
Specify what measurements are to be made.
Specify factors that can be controlled.
Suggest what uncontrollable factors may affect results.
Ensure that the experiment is feasible (within constraints) and ethical.
Carry out the experiment and analyse the results.

The experimenter is ultimately responsible for the design of the experiment.
AS3G / STM50
9
Role of the Statistician
Determine whether and how the hypotheses can be tested.
Are the proposed measurements suitable?
Do they satisfy the assumptions of the appropriate statistical tests to be used?
Are they likely to be reliable and consistent?
How many units should be measured?
Are the units independent?
How should treatments be allocated to units?
How can unexplained (residual) variation be minimised?
Try to keep the experiment as simple as possible, and an appropriate size.
The statistician generally acts as advisor to the experimenter, both in design and
analysis.
AS3G / STM50
10
Objectives
In order to design an experiment and complete the analysis the objective/s must
be clear.
Common mistakes in defining objectives are that they are too vague for
example:
To evaluate 4 improved mango varieties.

But this doesnt tell us:
Which varieties?
Under what conditions (location, management) are they to be evaluated?
What criteria will be used to evaluate them?
Linked to what problem solving strategy?
What is the science behind this?

Or they are trying to address too many objectives in a single trial (for example
both biophysical and farmer objectives).
AS3G / STM50
11
Experimental Units

The experimental units are the entities which are to be treated and measured.

A unit can be an individual person, animal, plant or inanimate object (e.g. a
machine). Or a unit may consist of many individuals, (e.g. a plot of wheat, Petri
dish of bacteria, or a household).

The units chosen to be studied in the experiment should be random samples
from the population of interest.

AS3G / STM50
12
Treatments

Treatments are the manipulations that we perform on the experimental units.
Generally we are interested in the average effects of the treatments and in
particular how these treatment means differ from each other.

Examples:

Experiment type Treatment
Medical Drug
Animal Diet
Agricultural Wheat Variety
Chemical Catalyst
Psychological Teaching Method



AS3G / STM50
13



Treatment may be levels of a single factor (e.g. various amounts of fertiliser
given, type of drug administered). But often treatments consist of a combination
of several factors (e.g. tomato variety, amount of water given, early/late sowing,
etc). An experiment using such treatments is called a Factorial Experiment,
and enables us to investigate any interactions between factors.

We often want to compare new treatments with an existing treatment (or no
treatment). Then one level of treatment acts as a control, so that one aim of the
experiment is to compare the other treatment levels with the control.

Normally each experimental unit will receive a single treatment.

AS3G / STM50
14
Randomisation

One important concern of experimental design is how best to allocate treatments
to units.

Some degree of randomisation is vital for the results of an experiment to be
meaningful.

For example, in a medical experiment where a drug is being compared with a
placebo, if the doctors are not made aware of the importance of randomisation,
they may be tempted to give the drug to those patients suffering the most severe
symptoms.

This will bias the results.

Randomisation, therefore, reduces bias and allows a fair comparison to be
made between treatments.

Randomisation also justifies the assumption of independent errors in the linear
model.
AS3G / STM50
15
Replication

Replication defines how many units receive the same treatment level.

Increasing the replication improves the accuracy of results and is essential to
provide estimates of s
2
and standard errors. It generally increases the power of
the experiment to detect important differences between treatments.

However, too much replication is a waste of resources, and may blow the
budget!

So a compromise is needed.

AS3G / STM50
16
Blocking

The purpose of an experiment is to investigate the effects of treatments and their
differences. But these can be obscured by the effects of other factors outside the
experimenters control, which cause the units to be heterogeneous.

Blocking is used to take account of differences (or heterogeneities) in the
experimental units (e.g. male/female, old/young patients, different fertility in
parts of a field).

Use of blocking can reduce the variability of the observations and so often
reduces the estimate of residual variance, s
2
.

Generally, randomisation has to be compromised with blocking, leading to some
form of restricted randomisation, (e.g. a randomised complete block design).

We will discuss blocking in more depth later in the module.

AS3G / STM50
17
Confounding

A variable (or several variables) that is related to both the dependant and
independent variables (or response and explanatory variables) is called a
confounding variable. Confounding variables need to be taken into account
when designing an experiment, to avoid the inferences from the analysis being
invalid.

For example: A design with 8 plots. Plots 1-4 have treatment A, and plots 5-8
have treatment B.
1 5
2 6
3 7
4 8
There is a clear difference between the yields for the two treatments. But is this
due to treatment? Or due to the fields for treatment A having a good water
supply? Or because the fields for treatment B are in the shade of the trees?

AS3G / STM50
18
An Example of the Story of an Experiment

t we know damage varies widely









Aim to find out if Boreproof is more resistant to stem borer
than M512.

Design 1: plant a field with Boreproof

Design 2: plant a field of Boreproof and another of M512
Observed damage (%):
Boreproof: 20
M512: 50
But we know damage varies widely

Design 3: plant 4 fields of Boreproof and 4 of M512
Observed damage (%):
Boreproof: 20, 10, 40, 30
M512: 50, 60, 40, 35
Objective
Comparison of
Treatments
Single Treatment Experimental Units
Replication
Precision
AS3G / STM50
19
An Example of the Story of an Experiment

t we know damage varies widely


Design 4: Unit = 10 fields x 2 plots in each.
Each treatment once in each field, M512 on the left hand plot
and Boreproof on the right.
Observed damage (%):

Field M512 Boreproof
1 50 20
2 30 10
3 20 10


Consistent differences but are they caused by treatment?

Design 5: as Design 4 but with random allocation of
treatments to units.

New Experimental
Units
Blocking
Allocation
Confounding
Randomisation