EXPERIMENTAL

DESIGN*

main source: Vernoy & Vernoy (1997) & Falik & Brown (1983)

SOME ESSENTIAL ISSUES PERTAINING TO

STATISTICAL ANALYSES PERFORMED ON
EXPERIMENTAL DATA
9 Were
W
the
th data
d t collected
ll t d ffrom a sample
l th
thatt ttrully
ll representing
ti
the population? What is the likelihood of a given outcome by
chance alone (probalility)?
9 How a well
well--controlled and wellwell-designed experiment
(research) looks like?
9 Suppose that I have run a wellwell-controlled and wellwell-designed
experiment and I observed a notable difference in behavior
between the groups (ie. samples). Is it valid to draw a direct
generalization of my
g
y findings
g to the entire p
population?
p
9 What are the statistical techniques to be employed that would
permit me to draw a valid generalization?

The main aims of this section is to

provide you with the basics and
essentials pertaining to the second
issues

How a well
well--controlled and well
well--designed
experiment (research) looks like?

IDENTIFYING VARIABLES
I d
d t Variables
V i bl
Independent
Independent variables are those that
are manipulated by the experimenter
and applied to the subject in order to
determine what effect they may have.
These are the factors that the
experimenter varies from one condition
to another, such as the items in
memory lists or the dosages of an
experimental drug.
In an experiment, a researcher may
want to compare the effects of two
different independent variables or the
effects of varying amounts (such as
different dosages of drugs) or levels
(such as low imagery or high imagery in
a memory experiment) of one particular
independent variable.

D
d tV
i bl
Dependent
Variables
Dependent variables are those that are
used to assess or measure the effects
of the independent variables.
In psychology, dependent variables are
measures of the behavior being
studied
studied.
The dependent variable depends on the
independent variable; if the variables
are causally related, the dependent
variable will vary, or change, as the
independent variable changes. Thus,
dependent variables are measured by
the experimenter to determine whether
the experimental procedures result in
any behavioral change.
Examples: the number of visual images
identified correctly (dependent) and the
amount of time it takes to push a button
(independent).

S
bj t V
i bl
Subject
Variables
Subject variables are those that describe subjects'
characteristics or attributes
attributes, factors that cannot be
manipulated by the experimenter.
Examples of subject variables are gender, IQ,
ethnicity age
ethnicity,
age, etc
etc.

S
bj t V
i bl
Subject
Variables
Subject variables are those that describe subjects'
characteristics or attributes
attributes, factors that cannot be
manipulated by the experimenter.
Examples of subject variables are gender, IQ,
ethnicity age
ethnicity,
age, etc
etc.

The experiment is to test the effect of lovastatin on the cholesterol

level (HDL) in the blood.
Patients

P01
P02
P03
P04
P05
P06
P07
P08
P09
P10
P11
P12
P13
P14
P15

x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
x12
x13
x14
x15

y1
y2
y3
y4
y5
y6
y7
y8
y9
y10
y11
y12
y13
y14
y15

where
X

the total amount of lovastatin (in mg) taken by patients (independent variable)

the HDL level in the blood (dependent variable)

Factors Affecting An
Experimental Design
The experimental design depends on:
F1

the n
number
mber of independent variables
ariables

F2

the number of levels within each independent

variable

F3

the way in which we assign subjects to the various

experimental conditions

F4

th samples
the
l ((size,
i
group))

Three Ways of Assigning Subjects to

Experimental Design (F1, F2, F3)
A between
between--subjects design (sometimes
called an independent-group design)
requires that each level of each
independent variable has different
subjects; thus, there is a distinct
difference between each level of the
experiment because each subject
participates in one and only one level.

A within
within--subjects design (also
called a repeated-measures
design) requires that each
subject participates in all levels
of all independent variables;
thus, each subject stays within
the experiment for its entire
run.

A mixed design most often occurs when there are at

least two independent variables and each subject
participates
ti i t in
i allll levels
l
l off one variable
i bl b
butt nott allll
levels of at least one of the other variables.

Example (for Clarification)

The aims of experiment:

To investigate possible techniques to improve his/her golf swing.
The techniques investigated (independent variables):
Mental imagery (visualizing each golf shot) and relaxation
techniques (specifically, deep breathing). Each of the
independent variables has two levels
levels: no imagery or imagery
and no relaxation or relaxation
relaxation.
The effect investigated
variables):
g
((dependent
p
)
The quality of swing
Th possible
The
ibl conditions
diti
off assigning
i i subjects
bj t tto experiment:
i
t
no-imagery/no-relaxation
g y
no-imagery/relaxation
imagery/no-relaxation
imagery/relaxation.

Fig 1: The general experimental design for the experiment to

Fig.1:
improve golf swing

Between--Subject Design
Between
Each
E
h subject
bj t participates
ti i t in
i only
l one off the
th
experimental conditions and is independent of the
other groups. The subjects in the no-imagery/norelaxation condition are different from the subjects
j
in the imagery/relaxation condition, and so forth. In
this way, subject effects such as innate athletic
ability are randomized throughout all conditions.

Fig.2: The subject assignment for a between-subject

design. Each subject participates in only one
experimental condition.

A between
between--subjects design
(sometimes called an
independent-group design)
requires that each level of each
independent variable has different
subjects; thus, there is a distinct
difference between each level of
the experiment because each
subject participates in one and
onlyy one level.

Between-Subject Design (contd)

BetweenAdvantages:
9 It is not affected by order effects where, as subjects participate
in several conditions, they tend to improve with practice until
they are considerably more adept in the final condition than
th were in
they
i the
th initial
i iti l condition.
diti
9 It is statistically the simplest and is therefore easier to analyze.
Disadvantages:
9 It requires more subjects. If you want to use six subjects in
each condition,
design.
condition you need 24 subjects to complete your design

Within--Subject Design
Within
Each subject participates in every condition. Thus,
in order to have six subjects in each condition, you
need a total of only six subjects in the entire
experiment.

Fig 3: The subject assignment for a within-subject

Fig.3:
within subject design.
design
Each subject participates in all experimental
conditions.

A within
within--subjects design (also
p
called a repeated-measures
design) requires that each
subject participates in all levels
of all independent variables;
thus, each subject stays within
the experiment
p
for its entire
run.

Within--Subject Design (contd)

Within
Advantages:
9 Subject effects, at least theoretically, may be eliminated or subtracted out.
9 Experiments using this design tend to be more sensitive to differences
between experimental conditions
conditions. Specically for our example
example, the same
subject participates in each of the relaxation/imagery conditions, so each
group, or cell, will have subjects of equally good or equally poor ability. It is not
affected by order effects where, as subjects participate in several conditions,
they tend to improve with practice until they are considerably more adept in
the final condition than they were in the initial condition.
Disadvantages:
9 The difficulty in controlling learning factors. Thus, the within-subjects design
should not be used when the subject's behavior is likely to change with the
practice or knowledge gained through previous conditions. For instance, by the
time the subjects
j
p
participate
p
in the fourth condition, they
y may
y all score higher
g
than they did in the first condition, merely because of practice effects.
9 After participating in a couple of conditions, subjects may figure out the purpose
of the experiment and change their responses accordingly. This is sometimes a
problem especially in experiments that
problem,
that, by their nature
nature, require a certain amount
of deception.

Mixed Design
A mixed design is used when it is
necessary to have each subject
participate
p
p
in all levels of one
independent variable,
variable while participating
in only one level of another independent
variable. For example, if we were to use
a mixed design
g in the g
golfing
g study,
y, we
could assign a particular subject (S1) to
only the no-relaxation conditions
(Fig.4A). Then this subject participates
solely
y in the no-relaxation/no-imagery
g y
and the no-relaxation/imagery
conditions. Or we could assign the
subject to only the two no-imagery
conditions ((Fig.4B).
g )
Mixed designs use fewer subjects than
between-subjects designs, but they are
y statistically.
y
the most difficult to analyze

Fig. 4

Two possible subject assignments for a mixed

design Each subject participates in all levels
design.
of one independent variable but in only one
level of the other independent variable.

Addressing
One-Group
g F4: Onep
Experimental Designs
The simplest
Th
i l t experimental
i
t l design
d i that
th t iinvolves
l
comparing a single sample mean to the mean of a known
population.
Example:
Suppose you want to know whether the fifth formers in
School X have better IQ than their colleagues in the
country. If you have access to national data for all fifth
formers IQ, you can give the IQ test to all fifth formers
in School X and compare
p
its mean to the mean of the
population.

Addressing F4: Completely

Randomized Designs
One of the simplest ways to compare more than one
sample.
I this
thi d
i th
i d
d t variable
i bl with
ith att least
l
t
In
design
there iis one independent
two different levels,
levels and subjects are selected and assigned
to one of the groups in a completely random fashion.
fashion

Experimental Design Using Completely Randomized

Design
The Experiment:
Suppose you are a human factors psychologist who has just been employed
by a large computer manufacturer
manufacturer. If you have ever used more than one type
of computer, you are probably aware that computers come equipped with a
variety of keyboards, each with its own distinctive touch and key positions.
You have been hired by the computer firm to determine which of two
keyboards is more efficient
efficient.
The Experimental Design:
The easiest way to do this it to conduct an experiment comparing one
keyboard to another by using a completely randomized design. First, you need
to devise or acquire a typing test that will provide a fair assessment of the two
keyboards. Scores on this test will be the dependent variable. The type of
keyboard used during the typing test will be the independent variable. All other
variables-room illumination, noise level, computer and monitor used, wordprocessing program, and so on should remain constant. You select subjects
randomly from a population of possible computer users, and then randomly
assign
ass
g tthem
e to o
one
eo
of tthe
e ttwo
og
groups
oups (see Fig.
g 5)
5). This
s ttwo-group
o g oup des
design
g will
enable you to determine which of two keyboards is the more efficient.

Fig. 5

A two-group completely randomized design having

one independent variable

The Experiment:
(if you want to test more than two keyboards).
The Experimental Design:
You can certainly test more than two keyboards by adding more groups by
randomly selecting more subjects and assigning them in a random fashion to
the additional groups. But adding more keyboards simply extends the
completely randomized design; it does not increase the number of
independent variables. There is still only one, the type of keyboard (see Fig.
6) You have merely increased the number of levels (number of different
6).
keyboards) of the independent variable.

Fig. 6

A completely randomized design with three

levels of one independent variable