0 Votes +0 Votes -

12 vues24 pagesEstadistica Permanova

Jan 16, 2017

© © All Rights Reserved

PDF, TXT ou lisez en ligne sur Scribd

Estadistica Permanova

© All Rights Reserved

12 vues

Estadistica Permanova

© All Rights Reserved

- Neuromancer
- The E-Myth Revisited: Why Most Small Businesses Don't Work and
- How Not to Be Wrong: The Power of Mathematical Thinking
- Drive: The Surprising Truth About What Motivates Us
- Chaos: Making a New Science
- The Joy of x: A Guided Tour of Math, from One to Infinity
- How to Read a Person Like a Book
- Moonwalking with Einstein: The Art and Science of Remembering Everything
- The Wright Brothers
- The Other Einstein: A Novel
- The 6th Extinction
- The Housekeeper and the Professor: A Novel
- The Power of Discipline: 7 Ways it Can Change Your Life
- The 10X Rule: The Only Difference Between Success and Failure
- A Short History of Nearly Everything
- The Kiss Quotient: A Novel
- The End of Average: How We Succeed in a World That Values Sameness
- Made to Stick: Why Some Ideas Survive and Others Die
- Algorithms to Live By: The Computer Science of Human Decisions
- The Universe in a Nutshell

Vous êtes sur la page 1sur 24

discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/227626391

analysis of variance

Article in Austral Ecology February 2001

DOI: 10.1111/j.1442-9993.2001.01070.pp.x

CITATIONS

READS

4,490

2,189

1 author:

M. J. Anderson

Massey University

117 PUBLICATIONS 21,322 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Functional biodiversity of New Zealand's marine fishes versus depth View project

All content following this page was uploaded by M. J. Anderson on 04 September 2014.

The user has requested enhancement of the downloaded file. All in-text references underlined in blue are added to the original document

and are linked to publications on ResearchGate, letting you access and read them immediately.

of variance

MARTI J. ANDERSON

Centre for Research on Ecological Impacts of Coastal Cities, Marine Ecology Laboratories A11,

University of Sydney, New South Wales 2006, Australia

Abstract Hypothesis-testing methods for multivariate data are needed to make rigorous probability statements

about the effects of factors and their interactions in experiments. Analysis of variance is particularly powerful for

the analysis of univariate data. The traditional multivariate analogues, however, are too stringent in their assumptions

for most ecological multivariate data sets. Non-parametric methods, based on permutation tests, are preferable.

This paper describes a new non-parametric method for multivariate analysis of variance, after McArdle and

Anderson (in press). It is given here, with several applications in ecology, to provide an alternative and perhaps

more intuitive formulation for ANOVA (based on sums of squared distances) to complement the description provided by McArdle and Anderson (in press) for the analysis of any linear model. It is an improvement on previous

non-parametric methods because it allows a direct additive partitioning of variation for complex models. It does

this while maintaining the flexibility and lack of formal assumptions of other non-parametric methods. The teststatistic is a multivariate analogue to Fishers F-ratio and is calculated directly from any symmetric distance or

dissimilarity matrix. P-values are then obtained using permutations. Some examples of the method are given for

tests involving several factors, including factorial and hierarchical (nested) designs and tests of interactions.

Key words: ANOVA, distance measure, experimental design, linear model, multifactorial, multivariate dissimilarity,

partitioning, permutation tests, statistics.

INTRODUCTION

The analysis of multivariate data in ecology is becoming increasingly important. Ecologists often need to test

hypotheses concerning the effects of experimental

factors on whole assemblages of species at once. This

is important for core ecological research and in studies

of biodiversity or environmental impacts in many

habitats, including marine subtidal environments

(Warwick et al. 1988; Gray et al. 1990; Chapman et al.

1995; Glasby 1997), mangroves (Skilleter 1996;

Kelaher et al. 1998), freshwater systems (Faith et al.

1995; Quinn et al. 1996) and terrestrial systems (Oliver

& Beattie 1996; Anderson & Clements, in press).

Univariate analysis of variance (ANOVA) provides an

extremely powerful and useful tool for statistical tests

of factors and their interactions in experiments

(Underwood 1981, 1997). Partitioning variation, as in

multifactorial ANOVA, is particularly important for testing hypotheses in complex ecological systems with natural temporal and spatial variability. This partitioning

is also needed to test multivariate hypotheses in ecology for experimental designs involving several factors.

*Present address: Department of Statistics, University of

Auckland, Private Bag 92019, Auckland, New Zealand (Email:

mja@stat.auckland.ac.nz).

Accepted for publication March 2000.

the general multivariate hypothesis of differences in the

composition and/or relative abundances of organisms

of different species (variables) in samples from different groups or treatments. This test is a significant

advance on previous methods because it can be based

on any measure of dissimilarity and can partition

variation directly among individual terms in a multifactorial ANOVA model. The test is applicable to any

situation where the simultaneous responses of many

potentially non-independent variables (usually abundances of species in an assemblage) have been measured in samples from a one-factor or multifactorial

ANOVA design.

Powerful multivariate statistical methods, such as the

traditional multivariate analysis of variance (MANOVA),

have existed for decades (Hotelling 1931; Wilks 1932;

Fisher 1936; Bartlett 1939; Lawley 1939; Pillai 1955),

but tests using these statistics rely on assumptions that

are not generally met by ecological data. The assumption that the data conform to a multivariate normal distribution is particularly unrealistic for most ecological

data sets. This is because the distributions of abundances of individual species are usually highly aggregated or skewed (e.g. Gaston & McArdle 1994). Also,

abundances take discrete values rather than being

continuous, species with small means often have

asymmetric distributions because they are necessarily

zeros to the data set. MANOVA test statistics are not

particularly robust to departures from multivariate normality (Mardia 1971; Olson 1974; Johnson & Field

1993). Finally, many of these test statistics are simply

impossible to calculate when there are more variables

than sampling units, which often occurs in ecological

applications.

Many non-parametric methods for tests of differences among a priori groups of observations (as in

MANOVA) have been developed (Mantel 1967; Mantel

& Valand 1970; Hubert & Schultz 1976; Mielke et al.

1976; Clarke 1988, 1993; Smith et al. 1990; Excoffier

et al. 1992; Edgington 1995; Pillar & Orlci 1996;

Legendre & Anderson 1999). These methods generally have two things in common. First, they are based

on measures of distance or dissimilarity between pairs

of individual multivariate observations (which I will

refer to generally as distances) or their ranks. A statistic is constructed to compare these distances among

observations in the same group versus those in different groups, following the conceptual framework of

ANOVA. Second, they use permutations of the observations to obtain a probability associated with the null

hypothesis of no differences among groups.

These non-parametric methods generally fall into

two categories. First, there are those that can be based

on any chosen distance measure. There are many such

measures and these have different properties, which

make them appropriate for different kinds of data

(Legendre & Legendre 1998). For example, to express

differences in community structure, the semimetric

BrayCurtis measure of ecological distance (Bray &

Curtis 1957) or Kulczynskis (1928) semimetric measure are generally preferred over metric measures, like

Euclidean distance (Odum 1950; Hajdu 1981; Faith

et al. 1987; Clarke 1993). The methods that are flexible enough to be used with any such distance measure

(e.g. Mantel 1967; Hubert & Schultz 1976; Smith

et al. 1990; Clarke 1993) have much to recommend

them for this reason.

The drawback to using these methods is that they are

not able to cope with multifactorial ANOVA. That is, they

are not able to partition variation across the many

factors that form part of the experimental design.

Consequently, for most complex designs, one must

analyse data as one-way analyses in multiple subsets

within particular levels of factors. These multiple oneway analyses and qualitative interpretations of ordination plots are then used to infer something about

interactions or variability at different spatial scales (e.g.

Anderson & Underwood 1994; Kelaher et al. 1998).

Some of the proposed non-parametric methods do

allow partitioning for a complex design (e.g. Excoffier

et al. 1992; Edgington 1995; Pillar & Orlci 1996), but

these are restricted for use with metric distance measures, which are not ideal for ecological applications.

33

there has been disagreement concerning appropriate

permutational strategies for complex ANOVA, particularly for tests of interactions (e.g. Edgington 1995;

Manly 1997). There have been some recent examples

of direct statistical analyses of BrayCurtis distances

(Faith et al. 1995; Underwood & Chapman 1998).

These are restricted, however, to very specific experimental designs or hypotheses and cannot be used for

any multifactorial ANOVA design.

Ecologists need a non-parametric multivariate

method that can partition variation based on any distance measure in any ANOVA design. The method needs

to be robust, interpretable by reference to the experimental design, and should lack formal assumptions

concerning distributions of variables. The purpose of

this paper is to outline just such a method and to give

some ecological examples of its use. The more general

mathematical theory underlying this method, along

with simulations and a comparison with the related

approach of Legendre and Anderson (1999), is

described elsewhere (McArdle & Anderson, in press).

MULTIVARIATE ANALYSIS

An outline for a general approach to the analysis of

multivariate data in ecology was given by Clarke and

Green (1988) and Clarke (1993). For experimental

designs used to test hypotheses defined a priori, there

are essentially four steps: (i) a choice is made concerning an appropriate transformation and/or standardization (if any) to apply to the data, given the hypothesis

and the scales and nature of the species variables; (ii)

a choice is made concerning the distance measure to

be used as the basis of the analysis (e.g. BrayCurtis,

Euclidean, x2 or other measure); (iii) ordination

(and/or clustering) is performed in order to visualize

patterns of resemblance among the observations based

on their community composition; and (iv) a nonparametric multivariate test for differences among

groups is done to obtain a rigorous probabilistic

statement concerning multivariate effects of a priori

groups. Note that (iii) is not essential in terms of

the statistical test; ordination simply gives a visual

representation by reducing the dimensionality of the

data. In this paper, I focus on step (iv) of this procedure, which currently poses a problem for multifactorial designs.

DESIGN

The two essential considerations for the test are: (i) the

construction of the test-statistic, and (ii) the calculation

34

M. J. ANDERSON

describe the method, which I shall simply call nonparametric MANOVA, first for the one-way design and

then for more complex designs, followed by some ecological examples. I deal here only with the case of

balanced ANOVA designs, but analogous statistics for any

linear model, including multiple regression and/or

unbalanced data, can be constructed, as described by

McArdle and Anderson (in press).

geometrically (e.g. Calinski & Harabasz 1974; Mielke

et al. 1976; Edgington 1995; Pillar & Orlci 1996), as

shown in Fig. 1 for the case of two groups and two variables (dimensions). Here, SSW is the sum of the

squared Euclidean distances between each individual

replicate and its group centroid (the point corresponding to the averages for each variable, Fig. 1 and

The essence of analysis of variance is to compare variability within groups versus variability among different

groups, using the ratio of the F-statistic. The larger the

value of F, the more likely it is that the null hypothesis

(H0) of no differences among the group means (i.e.

locations) is false. For univariate ANOVA, partitioning

of the total sum of squares, SST, is achieved by calculating sums of squared differences (i) between individual replicates and their group mean (SSW, the

within-group sum of squares; Table 1a), and (ii)

between group means and the overall sample mean

(SSA, the among-group sum of squares). Next, consider

the multivariate case where p variables are measured

simultaneously for each of n replicates in each of a

groups, yielding a matrix of data where rows are observations and columns are variables. A natural multivariate analogue may be obtained by simply adding up

the sums of squares across all variables (Table 1b). An

F-ratio can then be constructed, as in the univariate

case.

groups in two dimensions where the groups differ in location.

The within-group sum of squares is the sum of squared distances from individual replicates to their group centroid. The

among-group sum of squares is the sum of squared distances

from group centroids to the overall centroid. () Distances

from points to group centroids; () distances from group

centroids to overall centroid; (q), overall centroid; (h), group

centroid; (d), individual observation.

Table 1. Calculations of within-group sums of squares for partitioning in (a) univariate ANOVA, (b) a multivariate analogue

obtained by summing across variables, (c) a multivariate analogue equivalent to (b) obtained using sums of squared Euclidean

distances, (d) the traditional MANOVA approach, which yields an entire matrix (W) of within-group sums of squares and cross

products, and (e) the partitioning using inter-point distances advocated here, equivalent to (b) and (c) if Euclidean distances

are used

Univariate

(a) One variable

Multivariate

(b) Summed across variables

(c) Geometric approach

(inner product, a scalar, based on Euclidean distances, correlations between

variables ignored)

(d) Traditional MANOVA

(outer product, a matrix, based on Euclidean distances, correlations between

variables matter)

(e) Inter-point geometric approach

(a scalar, based on any distance measure, correlations between variables ignored)

a

n

p

SSW 5 S i 5 1 S j 5 1 Sk 5 1 ( yijk yi.k)2

a

n

SSW 5 S i 5 1 S j 5 1 ( yij yi.)T( yij yi.)

a

n

W 5 S i 5 1 S j 5 1 ( yij yi.)( yij yi.)T

N1 N

1

2

SS1 5 n Si 5 1 Sj 5 i 1 1 d ij eij

yij, univariate observation of the jth replicate (j 5 1,, n) in the ith group (i 5 1,, a); yijk, observation of yij for the kth

variable (k 5 1,, p); yij, vector of length p, indicating a point in multivariate space according to p variables (dimensions) for

observation j in group i. A superscript )T9 indicates the transpose of the vector, bars over letters indicate averages and a dot

subscript indicates averaging was done over that subscripted variable.

geometric approach yields one value for each of SSW,

SSA and SST as sums of squared Euclidean distances.

This geometric approach gives sums of squares equivalent to the sum of the univariate sums of squares

(added across all variables) described in the previous

paragraph. This differs from the traditional MANOVA

approach, where partitioning is done for an entire

matrix of sums of squares and cross-products (e.g.

Mardia et al. 1979; Table 1d).

The key to the non-parametric method described

here is that the sum of squared distances between points

and their centroid is equal to (and can be calculated

directly from) the sum of squared interpoint distances

divided by the number of points. This important

relationship is illustrated in Fig. 2 for points in two

dimensions. The relationship between distances to

centroids and interpoint distances for the Euclidean

measure has been known for a long time (e.g.

Kendall & Stuart 1963; Gower 1966; Calinski &

Harabasz 1974; Seber 1984; Pillar & Orlci 1996;

Legendre & Legendre 1998; see also equation B.1 in

Appendix B of Legendre & Anderson 1999). What is

important is the implication this has for analyses

based on non-Euclidean distances. Namely, an

additive partitioning of sums of squares can be obtained

for any distance measure directly from the distance

matrix, without calculating the central locations of

groups.

for the calculation of (a) a distance matrix from a raw data

matrix and (b) a non-parametric MANOVA statistic for a

one-way design (two groups)

directly from the distance

matrix. SST, sum of squared

distances in the half matrix

(j) divided by N (total

number of observations); SSW,

sum of squared distances

within groups ( ) divided by

n (number of observations

per group). SSA 5 SST SSW

and F = [SSA/(a 1)]/[SSW/

(N a)], where a 5 the number of groups.

35

based on Euclidean distances, the average for each variable across the observations within a group constitutes

the measure of central location for the group in

Euclidean space, called a centroid. For many distance

measures, however, the calculation of a central location

may be problematic. For example, in the case of the

semimetric BrayCurtis measure, a simple average

across replicates does not correspond to the central

location in multivariate BrayCurtis space. An

appropriate measure of central location on the basis

of BrayCurtis distances cannot be calculated

easily directly from the data. This is why additive

points to their centroid is equal to the sum of squared interpoint distances divided by the number of points.

36

M. J. ANDERSON

groups) has not been previously achieved using

BrayCurtis (or other semimetric) distances. However,

the relationship shown in Fig. 2 can be applied to

achieve the partitioning directly from interpoint

distances.

Thus, consider a matrix of distances between every

pair of observations (Fig. 3a). If we let N 5 an, the total

number of observations (points), and let dij be the distance between observation i 5 1,, N and observation

j 5 1,, N, the total sum of squares is

1 N1

S

SST 5

N i5l

j 5 i 11

d ij

(1)

the subdiagonal (or upper-diagonal) half of the distance

matrix (not including the diagonal) and divide by N

(Fig. 3b). In a similar fashion, the within-group or

residual sum of squares is

SSW 5

1 N1

S

n i5l

d ij e ij

j 5 i 11

(2)

where eij takes the value 1 if observation i and observation j are in the same group, otherwise it takes the

value of zero. That is, add up the squares of all of the

distances between observations that occur in the same

group and divide by n, the number of observations per

group (Fig. 3b). Then SSA 5 SST SSW and a pseudo

F-ratio to test the multivariate hypothesis is

SSA /(a 1)

F5

SSW /(N a)

(3)

If the points from different groups have different central locations (centroids in the case of Euclidean distances) in multivariate space, then the among-group

distances will be relatively large compared to the withingroup distances, and the resulting pseudo F-ratio will

be relatively large.

One can calculate the sums of squares in equations

(1) and (2) and the statistic in equation (3) from a

distance matrix obtained using any distance measure.

The statistic in equation (3) corresponds exactly to

the statistic in equation (4) of McArdle and Anderson

(in press), who have shown more generally how

partitioning for any linear model can be done directly

from the distance matrix, regardless of the distance

measure used. Another important aspect of the statistic described above is that, in the case of a Euclidean

distance matrix calculated from only one variable,

equation (3) gives the same value as the traditional

parametric univariate F-statistic.

This is proposed as a new non-parametric MANOVA

statistic that is intuitively appealing, due to its analogy

with univariate ANOVA, and that is extremely relevant

for ecological applications. The results (in terms of

sums of squares, mean squares and pseudo F-ratios)

obtained for individual terms in a multivariate analysis

for univariate ANOVA. The difference is that the hypothesis being tested for any particular term is a multivariate

hypothesis.

PERMUTATIONS

The multivariate version of the F-statistic described

here is not distributed like Fishers F-ratio under the

null hypothesis. This is so because (i) we do not expect

the individual variables to be normally distributed, and

(ii) we do not expect that the Euclidean distance will

necessarily be used for the analysis. Even if each of the

variables were normally distributed and the Euclidean

distance used, the mean squares calculated for the multivariate data would not each consist of sums of independent x2 variables, because, although individual

observations are expected to be independent, individual species variables are not independent of one

another. Thus, traditional tabled P-values cannot be

used. A distribution of the statistic under the null

hypothesis can be created, however, using permutations

of the observations (e.g. Edgington 1995; Manly

1997). The only situation in which one could use the

traditional tabled probabilities would be if one had a

single variable that could reasonably be assumed to be

normally distributed and one used Euclidean distances

for the analysis.

Suppose the null hypothesis is true and the groups

are not really different (in terms of their composition

and/or their relative abundances of species, as measured

by the BrayCurtis distances). If this were the case,

then the multivariate observations (rows) would be

exchangeable among the different groups. Thus, the

labels on the rows that identify them as belonging to a

particular group could be randomly shuffled (permuted) and a new value of F obtained (called, say, F p).

This random shuffling and re-calculation of F p is then

repeated for all possible re-orderings of the rows relative to the labels. This gives the entire distribution of

the pseudo F-statistic under a true null hypothesis for

our particular data. Comparing the value of F obtained

with the original ordering of the rows to the distribution

created for a true null by permuting the labels, a P-value

is calculated as

P5

(No. of F p F)

(Total no. of F p)

(4)

to be a member of the distribution of F p under permutation (i.e. it is one of the possible orderings of the

labels on the rows). The usual scientific convention of

an a priori significance level of a 5 0.05 is generally

used for interpreting the significance of the result, as

in other statistical tests. It is also possible to view the

hypothesis (Fisher 1955; Freedman & Lane 1983).

With a groups and n replicates per group, the number of distinct possible outcomes for the F-statistic in

a one-way test is (an)!/(a!(n!)a) (Clarke 1993). As it is

usually not practical to calculate all possible permutations, because of the time involved, P can be calculated

using a large random subset of all possible permutations

(Hope 1968). However, the precision of the P-value will

increase with increasing numbers of permutations.

Generally, at least 1000 permutations should be done

for tests with an a-level of 0.05 and at least 5000 permutations should be done for tests with an a-level of

0.01 (Manly 1997).

ASSUMPTIONS

The only assumption of the test is that the observations

(rows of the original data matrix) are exchangeable

under a true null hypothesis. To assume exchangeability

under the null hypothesis is generally to assume that

the observations are independent and that they have

similar distributions (e.g. Boik 1987; Hayes 1996). By

similar distributions, I mean similar multivariate dispersions of points, not that the points are necessarily

multivariate normal. The test described here is a test

for differences in location (means or centroids) among

groups of multivariate observations based on the

chosen distance measure. Like its univariate counterpart, which is sensitive to heterogeneity of variances,

this test and its predecessors that use permutations, like

ANOSIM (Clarke 1993), will also be sensitive to differences in the dispersions of points, even if the locations

do not differ.

The sensitivity of ANOSIM to differences in dispersion

has been suggested as an advantage by Clarke (1993).

This is because it was introduced in the context of

detecting environmental impacts, for which detection

of differences of any kind between control and impacted

locations is very important for environmental reasons.

Here, I simply suggest that caution be exercised in

interpreting the results of tests of significance.

Determining if significant differences among groups

may be due to differences in dispersion versus differences in location (or some combination of the two) is

an important statistical and ecological issue. The use

of permutation tests to obtain P-values does not avoid

this issue.

A useful comparative index of multivariate dispersion

has been given by Warwick and Clarke (1993). Also, a

separate permutation test for significant differences in

multivariate dispersions (after removing effects of differences in location), as an accompaniment to the nonparametric MANOVA approach given here, will be

described elsewhere (Anderson, Dutilleul, Lapointe &

Legendre, unpubl. data).

37

STATISTICS

Although the statistic described here is sensitive to differences in the relative dispersion of points among

groups, it takes no account of the correlations among

variables. In traditional MANOVA, the test-statistics (such

as Wilks Lambda) use information contained in the

between-group and/or within-group sample variance

covariance matrices (e.g. Table 1d, see Olson 1974;

Johnson & Field 1993). The traditional MANOVA tests

assume not only that the variance for each variable

remains constant across different groups (i.e. the

points in different groups have similar scatter), they also

assume that the relationships among the variables (their

covariances or correlations) do not differ across groups.

These differences in the sensitivities of different

multivariate test statistics are shown diagrammatically

in Fig. 4 for two variables (two dimensions). Figure 4(a)

shows two groups that differ in their correlation structure, but not in their variances or location. Figure 4(b)

shows two groups that differ only in their dispersions,

but not in their correlation or location. Although all

MANOVA statistics are designed to test for differences in

location, the traditional statistics will also be sensitive

to differences in correlations (Fig. 4a) as well as differences in dispersion (Fig. 4b). The method of nonparametric MANOVA described here will only be sensitive

to differences in dispersion (Fig. 4b). The correlations

among variables play no role in the analysis. If differences in the relationships amongst variables form a

hypothesis of interest, then some other non-parametric

techniques may be relevant (e.g. Biondini et al. 1991;

Krzanowski 1993).

GRAZERS

Consider the following example, taken from an ecological study by Anderson and Underwood (1997). The

study was designed to test the hypothesis that grazing

by gastropods affects intertidal estuarine assemblages.

The experiment was done at an intertidal oyster farm

from January to July 1994 in Quibray Bay, south of

Sydney, New South Wales, Australia. Experimental

surfaces (10 cm 3 10 cm) were enclosed in cages to

exclude gastropod grazers, while other surfaces were

left open to grazing. A third treatment consisted of

caged areas where natural densities of grazers were

included. This was a control for the effect of the cage

itself on assemblages. There were n 5 20 surfaces in

each of the three treatments. The numbers of individuals of each of 21 taxa (invertebrates and algae)

colonizing each surface were recorded.

The rationale for increasing the severity of the transformation to increase the relative contribution of rare

38

M. J. ANDERSON

and Green (1988), is followed here. Note that the transformation is not done in an effort to make data conform to any assumptions of the analysis. In this

example, the data contained some species that occurred

on a very large relative scale of abundance (e.g.

Spirorbid worms occurred in the thousands), so the

data were transformed by taking double-square roots

before the analysis. To visualize the multivariate

patterns among observations, non-metric multidimensional scaling (MDS) was performed on the

BrayCurtis distances (Kruskal & Wish 1978), using

the PRIMER computer program. Non-parametric

MANOVA was then done on BrayCurtis distances, as

described in the previous section, using the computer

program NPMANOVA, written by the author in FORTRAN.

The number of possible permutations for the oneway test in the case of the grazing experiment is

9.6 3 1025. A random subset of 4999 permutations was

used (Fig. 5). In this case, the null hypothesis of no

differences among groups was rejected, as the observed

value was much larger than any of the values obtained

under permutation (Fig. 5, Table 2).

A POSTERIORI TESTS

As in univariate ANOVA where there is a significant result

in a comparison of 3 or more treatments, we may wish

to ask for the multivariate case: wherein does the significant difference lie? This can be done by using the

same test, given above for the one-way comparison of

groups, but where individual pair-wise comparisons

between particular groups are done. To continue with

the logic of the analogous univariate situation, we

can use a t-statistic (which is simply the square root of

the value of the F-statistic described above) for these

Table 2. Non-parametric MANOVA on BrayCurtis distances for assemblages of organisms colonizing intertidal surfaces in estuaries in three grazing treatments (grazers

excluded, grazers inside cages, and surfaces open to grazers)

Source

d.f.

SS

MS

Grazers

Residual

Total

2

57

59

18 657.65

14 520.89

33 178.54

9328.83

254.75

Comparison*

Open versus caged

Open versus cage control

Caged versus cage control

36.61 0.0002

8.071

3.268

6.110

0.0002

0.0002

0.0002

where (a) the groups differ in correlation between variables,

but not in location or dispersion and (b) the groups differ

in dispersion, but not in location or correlation between

variables.

F-statistic for 4999 permutations of the data on assemblages

in different grazing treatments. The real value of F for these

data is very extreme by reference to this distribution

(F 5 36.62): thus there are strong differences among the

assemblages in different grazing treatments.

pairwise comparisons. These have the same interpretation as univariate t-tests, but they test the general

multivariate hypothesis of no difference between the

groups on the basis of the BrayCurtis (or other

chosen) distances. This is Students univariate

t-statistic if Euclidean distances are chosen for the

analysis of only one variable. P-values for each test are

obtained using separate sets of permutations that are

only done across the pair of groups being compared.

In this example, a random subset of 4999 permutations

was used (out of a possible 6.8 3 109) for each pairwise comparison.

For the analysis of the experimental removal of grazers, there was a significant difference among all pairs

of treatments: assemblages colonizing surfaces in cages

differed from those in the open or in cage controls

(t 5 8.07, P 5 0.0002 and t 5 6.11, P 5 0.0002, respectively). Grazers had a significant effect on the assemblages, which is consistent with the pattern of

separation of points corresponding to different treatments in the non-metric MDS plot (Fig. 6). The fact

that assemblages on open surfaces also differed significantly from those in cage controls (t 5 3.27,

P 5 0.0002) suggested that there was some additional

artifact due to the presence of a cage in the experiment.

An important point here is that the a posteriori comparisons just described did not make any correction for

experiment-wise error rate (Day & Quinn 1989).

Similarly, the multivariate pair-wise tests available in

the computer program NPMANOVA are not corrected for

experiment-wise error rate. This means that with an a

priori significance level of a 5 0.05, one should expect

to obtain a significant result in one out of every 20

39

P-value obtained under permutation for any individual

pair-wise test is exact. Many of the methods used for

correcting error rates for multiple comparisons, such

as the Bonferroni method, are very conservative but

may be applied.

For more complex designs, we can start by considering the situation with two factors in a factorial (or

orthogonal) design. The principles used to partition the

variation directly from the distance matrix and to obtain

the statistics and permutation methods for individual

terms in the model can be readily generalized to other

multifactorial cases. The logic applied to multifactorial

ANOVA of univariate data (e.g. see Underwood 1981,

1997) also applies to the analysis of multivariate data

using this non-parametric procedure. For example,

tests of main effects should be examined after tests for

interactions.

groups) and B designate factor 2 with b levels, with n

replicates in each of the ab combinations of the two factors. The total number of observations is N 5 abn. The

total sum of squares in the analysis is calculated as for

the one-way case according to equation (1). To partition

the variation, the within-group sum of squares for factor

A, ignoring any influence of B, is calculated as

SSW(A) 5

1 N1

S

bn i 5 1

d ij e(A)

ij

j 5 i 11

(5)

where e(A)

ij takes the value 1 if observation i and observation j are in the same group of factor A, otherwise it

takes the value of zero. Similarly, the within-group sum

of squares for factor B, ignoring any influence of A, is

1 N1

S

SSW(B) 5

an i 5 l

j 5 i 11

d ij e(B)

ij

(6)

the main effects in the analysis are SSA 5 SST SSW(A)

and SSB 5 SST SSW(B).

The residual sum of squares is calculated by considering the interpoint distances within each of the ab

combinations of factor A and B, thus:

SSR 5

Fig. 6. Non-metric MDS plot of assemblages colonizing

intertidal surfaces in Quibray Bay in each of three different

grazing treatments: (m), cage control; (s), open; ( ) caged.

1 N1

S

n i5l

j 5 i 11

d ij e(AB)

ij

(7)

where e(AB)

takes the value 1 if observation i and obserij

vation j are in the same combination of factors A and

B, otherwise it takes the value of zero. We then can

40

M. J. ANDERSON

interaction term: SSAB 5 SST SSA SSB SSR. It may

be easier to consider the squared distances being

summed in equations (5) through (7) by reference to

their physical location in the distance matrix itself, as

illustrated in Fig. 7.

In the case of a two-factor design where one factor

is nested in the other, the same general approach is

used. In this case, however, there is no interaction term

in the analysis and we have instead SSB(A) 5 SST SSA

SSR, where B(A) denotes that factor B is nested in

factor A.

Having obtained appropriate sums of squares, the

construction of the pseudo F-statistic for each term in

the analysis for non-parametric MANOVA then follows

the same rules and formulae as it would for the usual

univariate ANOVA. The construction of the F-ratio

depends on the experimental design, that is, whether

factors are nested or factorial and whether they are fixed

or random, exactly as for univariate ANOVA (e.g.

Underwood 1981, 1997; Winer et al. 1991).

test is not so simple if there is more than one factor in

the design. The choice of an appropriate permutation

method is not trivial and should be considered carefully for each term in the model. Indeed, the lack of

exact tests or knowledge of how the available approximate permutation tests might behave for complex

models has been a sticking point in the development

of multivariate non-parametric methods (e.g. Crowley

1992; Clarke 1993). To construct exact tests, two

important issues must be considered (Anderson & ter

Braak, unpublished data). First, which units should be

permuted (i.e. what are exchangeable under the null

hypothesis) and second, should any restrictions be

imposed on the permutations to account for other

factors in the design?

In many important situations, such as tests of interactions, no exact permutation test can be done. Also,

there are times when the exact test imposes so many

to partition the variability in the multivariate data set and to

calculate the sum of squares for each term in a two-factor

orthogonal design (each factor has two groups or levels). (a)

SSW(A) 5 sum of squared distances within groups of A (j),

divided by (bn). (b) SSW(B) 5 sum of squared distances

within groups of B ( ), divided by (an). (c) SSR 5 sum of

squared distances within combinations of AB ( ), divided

by (n) (residual sum of squares). SST 5 sum of squared

distances in the total half matrix, divided by (abn),

SSA 5 SST SSW(A), SSB 5 SST SSW(B), SSAB 5 SST SSA

SSB SSR.

there being too few possible permutations left. In these

cases, approximate permutation tests should be used,

of which there are several alternatives, including

permutation of residuals and permutation of raw data

across all terms in the analysis (e.g. Freedman & Lane

1983; ter Braak 1992; Manly 1997). Some empirical

comparisons of these methods are provided by

Gonzalez and Manly (1998) and Anderson and

Legendre (1999).

ECOLOGICAL EXAMPLES

Two-way factorial design

Harbour (part of Sydney Harbour) to test for the effect

of shade and proximity to the seafloor on assemblages

of invertebrates and algae colonizing subtidal hard surfaces near marinas (Glasby 1999). The experiment was

a two-way crossed (orthogonal) design with n 5 4 replicate settlement plates (15 cm 3 15 cm sandstone)

either far from or near to the seafloor (the factor of

position; all plates were at a similar depth of approximately 2 m below low water) and in one of three

shading treatments: (i) shade (an opaque plexiglass

roof), (ii) a procedural control (a clear plexiglass roof),

and (iii) no shade. Organisms colonizing the plates after

33 weeks were counted and a total of 46 taxa were

included in the analyses. Organisms that occurred

only once across the entire data set were not included.

Non-parametric MANOVA was done on BrayCurtis

distances calculated from double-root transformed data

using the FORTRAN program NPMANOVA. The sample

size was reasonably small for this study (n < 5), so the

41

data (e.g. Manly 1997; Gonzalez & Manly 1998) with

4999 random permutations. Similar results were

obtained using permutation of residuals under a

reduced model (not shown).

There was no significant interaction of shade and

position, but both main effects were significant

(Table 3, Fig. 8). It was then of interest to compare the

groups corresponding to different shading treatments

using a posteriori tests (Table 3). It was not necessary

to do this for the effect of position, because this factor

only had two groups. Assemblages of organisms on

settlement plates near the bottom were extremely different from those far away from the bottom (Fig. 8). Also,

assemblages on shaded plates were significantly different from those on either the procedural control or on

unshaded plates, which themselves did not differ

(Table 3, Fig. 8). This analysis also shows how the effect

of position relative to the bottom was much greater than

the effect of shading on assemblages in this experiment

(compare the values of their mean squares in Table 3).

The non-parametric approach advocated here allows

tests of significance, but it also allows relative sizes of

effects to be compared directly through the partitioning

of the variation and examination of mean squares.

the hypothesis that the size of a patch available for

Table 3. Non-parametric MANOVA on BrayCurtis distances for assemblages of organisms colonizing subtidal sandstone settlement panels after 33 weeks in an estuary at

different distances from the seafloor (positions either near or

far) and in three different shading treatments

Source

Position

Shade

Position 3

shade

Residual

Total

d.f.

1

2

2

18

23

SS

MS

5595.40 5595.40

3566.44 1783.22

1238.94 619.47

7440.66

17 841.43

Comparison*

Shade versus control

Shade versus no shade

Control versus no shade

13.536

4.314

1.499

0.0002

0.0006

0.1394

413.37

1.783

1.987

0.866

0.0154

0.0018

0.5560

assemblages colonizing sandstone settlement plates after

33 weeks in Middle Harbour that were either near to or far

from the seafloor and in one of three shading treatments. (r),

Shaded; ( ), control; (s), no shade; 1, far from the sea floor;

2, close to the sea floor.

42

M. J. ANDERSON

in equal areas on those patches (Anderson 1998). This

hypothesis was tested using wooden panels of three

different sizes (10 cm 3 10 cm, 20 cm 3 20 cm and

40 cm 3 40 cm). Two panels of a given size were

attached to sticks that were then strapped to the

structure of an intertidal oyster farm in the Port

Stephens estuary in New South Wales, Australia in

January of 1995 (see Anderson 1998 for details). Six

sticks (two for each of the three patch sizes) were then

collected independently after periods of 3, 6, 9, 12

and 18 months of exposure to colonization. The experimental design thus consisted of three factors: time

(5 periods of submersion), patch (3 sizes) and sticks

(2 sticks per time 3 patch combination, a random

nested factor), with n 5 2 panels per stick. Organisms

colonizing panels were quantified in a 10 cm 3 10 cm

area from each panel (chosen randomly from the largersized panels). A total of 33 taxa were included in

multivariate analyses.

The analysis was done using NPMANOVA on Bray

Curtis distances calculated on double-root transformed data, as for the previous examples. In this case,

however, an exact permutation test for the nested factor

(sticks) was carried out by permuting the observations

randomly across sticks, but only within the 5 3 3 combinations of levels of time 3 patch. Then, for the test

of the upper-level terms (the main effects of time, patch

size and their interaction), individual replicates on a

stick were permuted together as a unit (i.e. whole sticks

were permuted). This is done so that the upper-level

effects can be tested against the variability across sticks,

not across individual replicates, as is necessary under

the null hypothesis for a nested hierarchy (e.g. Clarke

1993). For all tests, a subset of 4999 permutations was

used.

The nested factor of sticks was not significant in the

multivariate analysis, but the time 3 patch size interaction was significant (Table 4). Individual pair-wise

comparisons of effects of patch size for each time were

nearly identical to the one-way tests given in Anderson

(1998) using analysis of similarities (ANOSIM, Clarke

1993). Assemblages were significantly different on the

smallest patches compared to the other sized patches

after 3, 6, 9 or 12 months. Assemblages on the two

larger sized patches did not differ significantly from one

another except after 12 months. After 18 months,

assemblages were similar on all patch sizes.

Non-metric MDS plots helped to interpret these

results. Two separate ordinations were done on these

data, as the stress value for the non-metric MDS

plot that included all observations was too high for a

reasonable interpretation. The effect of different patch

sizes appeared to be fairly consistent (in terms of

its magnitude and direction) after 3, 6 or 9 months

(Fig. 9a). After 12 or 18 months, the observations

become more scattered within and across the groups

and the effects of patch size become less clear (Fig. 9b).

Increased dispersion (variability in assemblages) after

these longer periods of time, compared to earlier

periods, is seen clearly in the two-factor plot of stick

centroids (Fig. 9c, which includes all data). As noted

earlier, the tests are sensitive to such differences in dispersion. Overall, although the two factors did interact,

the effect of time (i.e. succession) was relatively more

important in distinguishing assemblages than the size

of the patch (compare their mean squares in Table 4),

and effects of patch size decreased through time for

these assemblages (see Anderson 1998 for further

details).

DISCUSSION

Natural temporal and spatial variability is intrinsic to

ecological systems. Indeed, variability might be considered the currency of ecological scientific work. It is

for this reason that statistical analysis plays such an

important role in the development of ecology as a

science. In Design of Experiments, R. A. Fisher (1935,

p. 4) wrote:

We may at once admit that any inference from the

particular to the general must be attended with

some degree of uncertainty, but this is not the

same as to admit that such inference cannot be

absolutely rigorous, for the nature and degree of

uncertainty may itself be capable of rigorous

expression.

Quantitative statistical inference is indeed what is

needed for the rigorous interpretation of mensurative

or manipulative ecological experiments. Although our

conclusions may be uncertain, they are still rigorous in

the sense that the degree of uncertainty can be

expressed in terms of mathematical probability. In univariate analysis, W. S. Gosset (Student 1908) made

this possible for comparisons of two treatments, while

R. A. Fisher made this possible for many treatments

and experimental factors. In a complex and intrinsically variable world, ANOVA allows us to identify simultaneous effects and interactions of more than one

factor, and to identify the uncertainty of our inferences

with rigour (e.g. Underwood 1981, 1997).

An important advance in the analysis of multivariate

data in ecology was the development of non-parametric

methods for testing hypotheses concerning whole

communities (e.g. Clarke 1988, 1993; Smith et al.

1990; Biondini et al. 1991). Some parallel advances

were made in the context of tests for significant clusters in cluster analysis (e.g. Good 1982; Gordon 1994).

Before these applications, particularly ANOSIM (Clarke

1993), became widely available, most multivariate

analyses in ecology focused on the reduction of dimensionality to produce and interpret patterns (ordination

43

observations into natural groups (clustering). These

methods, although extremely useful towards their

purpose, do not rigorously express the nature and

degree of uncertainty concerning a priori hypotheses.

Methods like Mantels test (Mantel 1967), ANOSIM

(Clarke 1993) and multiresponse permutation procedures (Mielke et al. 1976) allow such rigorous

probabilistic statements to be made for multivariate

ecological data.

The drawback to such non-parametric tests is that

they cannot easily be extended to the multifactorial

designs so common now in ecological studies. Two

sticking points prevented this: (i) the lack of a generalized statistic for partitioning variation, and (ii) the lack

of appropriate permutation methods (e.g. Clarke 1993;

Legendre & Anderson 1999). Although traditional

test-statistics used for MANOVA allow partitioning,

their restrictive assumptions have prevented their

effective use in ecology. The method of distancebased redundancy analysis (Legendre & Anderson

Table 4. Non-parametric MANOVA on BrayCurtis distances for assemblages of organisms colonizing wooden

settlement panels of three different sizes after 3, 6, 9, 12 or

18 months on an intertidal oyster farm

Source

d.f.

Time

4

Patch Size

2

Time 3 patch 8

Sticks (time 3 15

patch)

Residual

30

Total

59

Comparison

SS

MS

6414.99 3207.49

8.68

6224.03 778.004 2.10

5544.66 369.64

1.28

8697.09

57 186.48

P

0.0002

0.0002

0.0062

0.3384

289.90

3

6

9

12

18

months months months months months

Small versus

2.24*

medium

Small versus

2.30*

large

Medium versus 1.49

large

1.86*

1.48*

2.00*

1.38

2.87*

2.38*

2.54*

3.02*

1.47

1.69

1.75*

1.43

within each time using the t-statistic. Sticks were ignored in

the pair-wise tests. There were 35 possible permutations for

each.

colonizing intertidal wooden settlement panels of three different sizes for (a) periods of 3, 6 or 9 months and (b) periods of 12 or 18 months for raw data and (c) where centroids

were plotted corresponding to each stick across all combinations of time 3 patch size. The points corresponding to

assemblages after particular times of submersion (numbers

indicate the period in months) have been outlined for clarity. (d), Small; (h), medium; (m), large.

44

M. J. ANDERSON

rather complicated steps and involves the use of a

correction constant to distances. Although this

correction does not adversely affect the test (generally

making it more conservative, if anything), accurate

P-values are not given by this method in the case of

anything other than a one-factor design (McArdle &

Anderson, in press).

The method presented here has, in some sense,

combined the best of both worlds. Like the traditional

test-statistics, it can partition variation according to

any ANOVA design. Like the most flexible nonparametric methods, it can be based on any symmetric dissimilarity or distance measure (or their

ranks) and provides a P-value using appropriate permutation methods. That is, one can still choose a relevant transformation and an appropriate distance

measure (or use ranks of distances), consistent with the

method of ordination used to visualize patterns. By

using permutations, the test requires no specific

assumption concerning the number of variables or the

nature of their individual distributions or correlations.

The statistic used is analogous to Fishers F-ratio and

is constructed from sums of squared distances (or

dissimilarities) within and between groups. Another

feature of this statistic is that it is equal to to Fishers

original F-ratio in the case of one variable and when

Euclidean distances are used.

It is perhaps important to point out that the

BrayCurtis measure of dissimilarity may or may not

be the most appropriate measure to use in any given

situation. A point still commonly ignored is that

BrayCurtis and related measures, such as Kulczynskis

coefficient, will tend to under-estimate true ecological

distances when distances become large (i.e. when

observations have very few species in common),

as determined by simulations (Faith et al. 1987;

Belbin 1991). The BrayCurtis measure may therefore

only be useful insofar as it will produce reasonable

ecological ordinations, through the ranks used for

MDS. In light of this issue, Faith (1990) proposed

doing MANOVA on ordination scores obtained from

MDS.

An analysis based only on ordination scores will omit

some portion of the ecological information contained

in the original distance matrix and will depend on the

number of dimensions chosen for the MDS (Clarke

1993). The severity of this potential problem will

obviously increase with increases in the stress value

associated with an MDS plot. The actual amount

and the kind of information lost in reducing dimensions

using MDS are generally unknown and will depend

on the particular data set. Also, any subsequent

statistical inferences on ordination scores (using

traditional MANOVA, as described in Faith 1990; or

based on permutation tests) would be limited to

points in the ordination plots, rather than being applicable to the original observations.

measure for ecological data. The method described

here may be used with any distance measure chosen

(or on ranks of distances). It is useful to have the flexibility to choose a distance measure appropriate for the

data and hypothesis being tested. Although the

BrayCurtis measure has now become commonplace

in ecological studies (perhaps due to its availability in

the PRIMER computer program, or due to its intuitive

interpretation as percentage difference, or due to the

results presented in Faith et al. 1987), there are still

many rivals. Over 60 measures of similarity or dissimilarity have been described, with very few actual comparisons of their performance with different kinds

of ecological data (see Lamont & Grant 1979;

Legendre & Legendre 1998). This is undoubtedly an

area needing further research.

The approach advocated here is that multifactorial

analysis of variance, as successfully applied to univariate

data in ecology (e.g. Underwood 1981, 1997), can and

should also be applied to multivariate data for testing

hypotheses in a logical and rigorous way. It stands to

reason that the developments in experimental design

for ecology that require multifactorial ANOVA, in order

to, for example, avoid pseudo-replication (Hurlbert

1984), test for generality (Beck 1997) or test for environmental impact (e.g. Green 1979, 1993; Underwood

1993; Glasby 1997), should be incorporated into

multivariate analysis. The method described here

allows that to happen, but within a framework that is

general enough to suit our need for few assumptions

and flexibility in the multivariate analysis of ecological

data.

ACKNOWLEDGEMENTS

I am indebted to those who have worked on and developed multivariate methods and permutation tests that

have led to the ideas in this manuscript, including

K. R. Clarke, E. S. Edgington, R. A. Fisher,

P. Legendre, B. F. J. Manly, N. Mantel, B. H. McArdle,

L. Orlci, V. D. P. Pillar, E. P. Smith, and C. J. F. ter

Braak. I also owe a great deal to A. J. Underwood for

his work in the statistical analysis of ecological experiments using ANOVA, which inspired my pursuit of this

topic for multivariate analysis. T. Glasby kindly

provided data for the two-way factorial example. The

PRIMER computer program was provided courtesy of

M. R. Carr and K. R. Clarke, Plymouth Marine

Laboratories, UK. My colleagues at the Special

Research Centre for Ecological Impacts of Coastal

Cities provided logistic support, tested out the computer program and commented on earlier versions of

the manuscript. The computer program NPMANOVA is

available from the author. This research was supported

by a U2000 Post-doctoral Fellowship at the University

of Sydney.

REFERENCES

Anderson M. J. (1998) Effects of patch size on colonisation in

estuaries: revisiting the species-area relationship. Oecologia

118, 8798.

Anderson M. J. & Clements A. (in press) Resolving environmental

disputes: a statistical method for choosing among competing

cluster models. Ecol. Applic.

Anderson M. J. & Legendre P. (1999) An empirical comparison

of permutation methods for tests of partial regression

coefficients in a linear model. J. Stat. Comput. Simul. 62,

271303.

Anderson M. J. & Underwood A. J. (1994) Effects of substratum

on the recruitment and development of an intertidal estuarine fouling assemblage. J. Exp. Mar. Biol. Ecol. 184, 21736.

Anderson M. J. & Underwood A. J. (1997) Effects of gastropod

grazers on recruitment and succession of an estuarine assemblage: a multivariate and univariate approach. Oecologia 109,

44253.

Bartlett M. S. (1939) A note on tests of significance in multivariate analysis. Proc. Camb. Philos. Soc. 35, 1805.

Beck M. W. (1997) Inference and generality in ecology: current

problems and an experimental solution. Oikos 78, 26573.

Belbin L. (1991) Semi-strong hybrid scaling, a new ordination

algorithm. J. Veg. Sci. 2, 4916.

Biondini M. E., Mielke P. W. & Redente E. F. (1991) Permutation

techniques based on Euclidean analysis spaces: a new and

powerful statistical method for ecological research. In:

Computer Assisted Vegetation Analysis (eds E. Feoli & L.

Orlci) pp. 22140. Kluwer Academic Publishers,

Dordrecht.

Boik R. J. (1987) The FisherPitman permutation test: a nonrobust alternative to the normal theory F test when variances

are heterogeneous. Br. J. Math. Stat. Psychol. 40, 2642.

Bray J. R. & Curtis J. T. (1957) An ordination of the upland forest

communities of southern Wisconsin. Ecol. Monogr. 27,

32549.

Caliski T. & Harabasz J. (1974) A dendrite method for cluster

analysis. Commun. Stat. 3, 127.

Chapman M. G., Underwood A. J. & Skilleter G. A. (1995)

Variability at different spatial scales between a subtidal assemblage exposed to discharge of sewage and two control sites.

J. Exp. Mar. Biol. Ecol. 189, 10322.

Clarke K. R. (1988) Detecting change in benthic community

structure. In: Proceedings XIVth International Biometric

Conference, Namur: Invited Papers, pp. 13142. Socit

Adolphe Quetelet, Gembloux.

Clarke K. R. (1993) Non-parametric multivariate analyses of

changes in community structure. Aust. J. Ecol. 18, 11743.

Clarke K. R. & Green R. H. (1988) Statistical design and analysis

for a biological effects study. Mar. Ecol. Prog. Ser. 46,

21326.

Crowley P. H. (1992) Resampling methods for computationintensive data analysis in ecology and evolution. Ann. Rev.

Ecol. Syst. 23, 40547.

Day R. W. & Quinn G. P. (1989) Comparison of treatments after

an analysis of variance. Ecol. Monogr. 59, 43363.

Edgington E. S. (1995) Randomization Tests, 3rd edn. Marcel

Dekker, New York.

Excoffier L., Smouse P. E. & Quattro J. M. (1992) Analysis of

molecular variance inferred from metric distances among

DNA haplotypes: application to human mitochondrial DNA

restriction data. Genetics 131, 47991.

Faith D. P. (1990) Multivariate methods for biological monitoring

based on community structure. In: The Australian Society of

45

Region Research Institute.

Faith D. P., Dostine P. L. & Humphrey C. L. (1995) Detection

of mining impacts on aquatic macroinvertebrate communities: results of a disturbance experiment and the design of a

multivariate BACIP monitoring program at Coronation Hill,

N. T. Aust. J. Ecol. 20, 16780.

Faith D. P., Minchin P. R. & Belbin L. (1987) Compositional

dissimilarity as a robust measure of ecological distance.

Vegetatio 69, 5768.

Fisher R. A. (1935) Design of Experiments. Oliver & Boyd,

Edinburgh.

Fisher R. A. (1936) The use of multiple measurements in taxonomic problems. Ann. Eugen. 7, 17988.

Fisher R. A. (1955) Statistical methods and scientific induction.

J. Roy. Stat. Soc. 17, 6978.

Freedman D. & Lane D. (1983) A nonstochastic interpretation

of reported significance levels. J. Bus. Econ. Stat. 1, 2928.

Gaston K. J. & McArdle B. H. (1994) The temporal variability

of animal abundances: measures, methods and patterns. Phil.

Trans. Roy. Soc. Lond. 345, 33558.

Glasby T. M. (1997) Analysing data from post-impact studies

using asymmetrical analyses of variance: a case study of

epibiota on marinas. Aust. J. Ecol. 22, 44859.

Glasby T. M. (1999) Interactive effects of shading and proximity

to the seafloor on the development of subtidal epibiotic

assemblages. Mar. Ecol. Prog. Ser. 190, 11324.

Gonzalez L. & Manly B. F. J. (1998) Analysis of variance

by randomization with small data sets. Environmetrics 9,

5365.

Good I. J. (1982) An index of separateness of clusters and a

permutation test for its significance. J. Stat. Comput. Simul.

15, 814.

Gordon A. D. (1994) Identifying genuine clusters in a classification. Comput. Stat. Data Anal. 18, 56181.

Gower J. C. (1966) Some distance properties of latent root and

vector methods used in multivariate analysis. Biometrika 53,

32538.

Gray J. S., Clarke K. R., Warwick R. M. & Hobbs G. (1990)

Detection of initial effects of pollution on marine benthos:

an example from the Ekofisk and Eldfisk oilfields, North Sea.

Mar. Ecol. Prog. Ser. 66, 28599.

Green R. H. (1979) Sampling Design and Statistical Methods for

Environmental Biologists. Wiley, New York.

Green R. H. (1993) Application of repeated measures designs in

environmental impact and monitoring studies. Aust. J. Ecol.

18, 8198.

Hajdu L. J. (1981) Graphical comparison of resemblance measures in phytosociology. Vegetatio 48, 4759.

Hayes A. F. (1996) Permutation test is not distribution free.

Psychol. Methods 1, 18498.

Hope A. C. A. (1968) A simplified Monte Carlo significance test

procedure. J. Roy. Stat. Soc. 30, 58298.

Hotelling H. (1931) The generalization of Students ratio. Ann.

Math. Stat. 2, 36078.

Hubert L. & Schultz J. (1976) Quadratic assignment as a general data analysis strategy. Br. J. Math. Stat. Psychol. 29,

190241.

Hurlbert S. H. (1984) Pseudoreplication and the design of ecological field experiments. Ecol. Monogr. 54, 187211.

Johnson C. R. & Field C. A. (1993) Using fixed-effects model

multivariate analysis of variance in marine biology and

ecology. Oceanogr. Mar. Biol. Ann. Rev. 31, 177221.

Kelaher B. P., Chapman M. G. & Underwood A. J. (1998)

Changes in benthic assemblages near boardwalks in

46

M. J. ANDERSON

228, 291307.

Kendall M. G. & Stuart A. (1963) The Advanced Theory of

Statistics, Vol. 1, 2nd edn. Charles Griffith, London.

Kruskal J. B. & Wish M. (1978) Multidimensional Scaling. Sage

Publications, Beverly Hills.

Krzanowski W. J. (1993) Permutational tests for correlation

matrices. Statistics and Computing 3, 3744.

Kulczynski S. (1928) Die Pflanzenassoziationen der Pieninen.

Bull. Int. Acad. Pol. Sci. Lett. Cl. Sci. Math. Nat. Ser. B,

(Suppl. II) 1927, 57203.

Lamont B. B. & Grant K. J. (1979) A comparison of twenty-one

measures of site dissimilarity. In: Multivariate Methods in

Ecological Work (eds L. Orloci, C. R. Rao & W. M. Stiteler)

pp. 10126. International Co-operative Publishing House,

Fairland.

Lawley D. N. (1939) A generalization of Fishers IX test.

Biometrika 30, 1807 (Corrections in Biometrika 30, 4679).

Legendre P. & Anderson M. J. (1999) Distance-based redundancy

analysis: testing multispecies responses in multifactorial ecological experiments. Ecol. Monogr. 69, 124.

Legendre P. & Legendre L. (1998) Numerical Ecology, 2nd English

edn. Elsevier Science, Amsterdam.

McArdle B. H. & Anderson M. J. (in press) Fitting multivariate

models to community data: a comment on distance-based

redundancy analysis. Ecology.

Manly B. F. J. (1997) Randomization, Bootstrap and Monte Carlo

Methods in Biology, 2nd edn. Chapman & Hall, London.

Mantel N. (1967) The detection of disease clustering and a

generalized regression approach. Cancer Res. 27, 20920.

Mantel N. & Valand R. S. (1970) A technique of nonparametric

multivariate analysis. Biometrics 26, 54758.

Mardia K. V. (1971) The effect of non-normality on some multivariate tests and robustness to nonnormality in the linear

model. Biometrika 58, 10521.

Mardia K. V., Kent J. T. & Bibby J. M. (1979) Multivariate

Analysis. Academic Press, London.

Mielke P. W., Berry K. J. & Johnson E. S. (1976) Multi-response

permutation procedures for a priori classifications. Commun.

Stat. Theory Methods 5 (14), 140924.

Odum E. P. (1950) Bird populations of the Highlands (North

Carolina) Plateau in relation to plant succession and avian

invasion. Ecology 31, 587605.

Oliver I. & Beattie A. J. (1996) Designing a cost-effective invertebrate survey: a test of methods for rapid assessment of biodiversity. Ecol. App. 6, 594607.

Olson C. L. (1974) Comparative robustness of six tests in multivariate analysis of variance. J. Am. Stat. Assoc. 69, 894908.

analysis. Ann. Math. Stat. 26, 11721.

Pillar V. D. P. & Orlci L. (1996) On randomization testing in

vegetation science: multifactor comparisons of relev groups.

J. Veg. Sci. 7, 58592.

Quinn G. P., Lake P. S. & Schreiber S. G. (1996) Littoral benthos

of a Victorian lake and its outlet stream: spatial and temporal variation. Aust. J. Ecol. 21, 292301.

Seber G. A. F. (1984) Multivariate Observations. John Wiley and

Sons, New York.

Skilleter G. A. (1996) Validation of rapid assessment of damage

in urban mangrove forests and relationships with Molluscan

assemblages. J. Mar. Biol. Ass. UK 76, 70116.

Smith E. P., Pontasch K. W. & Cairns J. (1990) Community

similarity and the analysis of multispecies environmental

data: a unified statistical approach. Water Res. 24, 50714.

Student. (1908) The probable error of a mean. Biometrika 6,

125.

ter Braak C. J. F. (1992) Permutation versus bootstrap significance tests in multiple regression and ANOVA. In:

Bootstrapping and Related Techniques (eds K. H. Jckel,

G. Rothe & W. Sendler) pp. 7986. Springer-Verlag,

Berlin.

Underwood A. J. (1981) Techniques of analysis of variance in

experimental marine biology and ecology. Oceanogr. Mar.

Biol. Ann. Rev. 19, 513605.

Underwood A. J. (1993) The mechanics of spatially replicated

sampling programmes to detect environmental impacts in a

variable world. Aust. J. Ecol. 18, 99116.

Underwood A. J. (1997) Experiments in Ecology: Their Logical

Design and Interpretation Using Analysis of Variance.

Cambridge University Press, Cambridge.

Underwood A. J. & Chapman M. G. (1998) A method for

analysing spatial scales of variation in composition of assemblages. Oecologia 107, 5708.

Warwick R. M., Carr M. R., Clarke K. R., Gee J. M. & Green

R. H. (1988) A mesocosm experiment on the effects of

hydrocarbon and copper pollution on a sublittoral softsediment meiobenthic community. Mar. Ecol. Prog. Ser. 46,

18191.

Warwick R. M. & Clarke K. R. (1993) Increased variability as a

symptom of stress in marine communities. J. Exp. Mar. Biol.

Ecol. 172, 21526.

Wilks S. S. (1932) Certain generalizations in the analysis of

variance. Biometrika 24, 47194.

Winer B. J., Broan D. R. & Michels K. M. (1991) Statistical

Principles in Experimental Design, 3rd edn. McGraw-Hill,

Sydney.

FORUM

of Field Studies With Design Constraints

LUIS FERNANDO CHAVES1,2

insects reects the complexity of working with living organisms: the nite and limited input of

individuals, their relatedness (ecological and/or genetic), and the need to group organisms into

functional experimental units to estimate population parameters (e.g., cohort replicates). Several

decades ago, when the issue of pseudoreplication was rst recognized, it was highlighted that

mainstream statistical tools were unable to account for the lack of independence. For example, the

variability as a result of differences across individuals would be confounded with that of the experimental units where they were observed (e.g., pans for mosquito larvae), whereas both sources of

variability now can be separated using modern statistical techniques, such as the linear mixed effects

model, that explicitly consider the different scales of variability in a dataset (e.g., mosquitoes and pans).

However, the perception of pseudoreplication as a problem without solution remains. This study

presents concepts to critically appraise pseudoreplication and the linear mixed effects model as a

statistical solution for analyzing data with pseudoreplication, by separating the different sources of

variability and thereby generating correct inferences from data gathered in studies with constraints

in randomization.

KEY WORDS linear mixed effects model, Culex quinquefasciatus, Anopheles nuneztovari, bootstrap,

data analysis

cited and misunderstood concepts in the statistical

analysis of ecological studies on insects and other

organisms. Pseudoreplication is dened as the use of

inferential statistics to test for treatment effects with

data from experiments where either treatments are

not replicated . . . or replicates are not statistically different . . . (Hurlbert 1984). This concept has been

very inuential and pervasive, to the extent that pseudoreplication is widely cited as a major aw of most

eld studies (Heffner et al. 1996). Hurlberts major

claim was correct, and he basically showed that mainstream statistical tools at that time (e.g., analysis of

variance) were not suitable for the analysis of most

experimental designs. However, the uncritical appraisal of his study has been a major barrier for the

publication of results and, therefore, the advancement

of ecology (Oksanen 2001). Hurlberts study did not

prevent the unsuitable analysis of valuable datasets or

the proliferation of unsound experimental practices,

such as the movement of sampling units to control for

spatial/temporal variability (Alto and Juliano 2001a,

2001b; Reiskind and Wilson 2004). As thoughtfully

1 Corresponding author: Department of Environmental Studies,

Emory University, 400 Dowman Drive, Suite E510, Atlanta GA 30322

(e-mail: lfchave@emory.edu).

2 Laboratorio de Biologa Teo

rica, Instituto de Zoologa y Ecologa

Tropical, Facultad de Ciencias, Universidad Central de Venezuela,

Caracas, Venezuela.

studies is not the application of statistical analysis to

ecological data per se, but rather its application to the

understanding of ongoing ecological phenomena from

variation of individual phenotypic traits to the assemblages of organisms in populations, communities, and

ecosystems. Unlike physics or chemistry, in which the

supply of individual objects of study is practically

unlimited, the objects of study for an entomologist (or

more generally a naturalist) are nite and constrained.

Thus, limitations in randomization will likely arise, and

the science of statistics has developed new solutions to

correctly analyze the lack of independence in eld

data since Hurlberts study (Millar and Anderson

2004). The current forum article presents the following: 1) key concepts of experimental design to critically appraise and demystify the concept of pseudoreplication; 2) linear mixed effects models

(LMEMs) as powerful tools to analyze data originated

from constrained designs or to produce more general

inferences from classical randomized designs (e.g.,

blocks); and 3) how these tools can be used to further

gain insights from the data that can strengthen our

understanding of insect ecology. In developing point

2, equations and a guide to interpret them as models

used to analyze common entomological data are presented, as well as the implementation of this type of

analysis in the open source software R.

292

Analysis

Experiments are one of the major tools for hypothesis testing (Fisher 1935). In general, the idea is to

subject individual units of observation to varying degrees of independent and/or controllable factor(s),

and to determine how the levels of variation explain a

given pattern (Box 1980). Field studies focus on the

impacts of natural (or controlled) variation in environmental factors on individual units of observation.

Hypothesis testing has been central to the development of modern science, to the point that hypothesisdriven experiments or eld studies are one of the most

prominent requirements for project support by funding agencies. One of the major reasons for the widespread appeal of hypothesis-driven experiments and

eld studies has been their close association with tools

for data analysis to determine the impact of different

independent variables. The best example of a statistical tool guiding experimental design is the use of the

linear model (LM). This model assumes that variability across a set of individual units of observation (yi)

is explained by a series of n independent variables (x1,

x2, . . . , xn) and by a unique source of unexplained

variability, normally referred to as error (). These

models are linear, because the parameters enter linearly into the equation that relates the independent

variables to the outcome (Faraway 2006, Chaves and

Pascual 2007). A major constraint of these models is

that they assume total independence among the subjects of study, i.e., individual observation units are

unrelated at least within strata (i.e., after accounting

for the explanatory variables), which is the formal

denition of replication. When there is a lack of

independence across objects of study (i.e., pseudoreplication), the use of LMs with a unique source of

variability is inappropriate, because the variability is

modeled incorrectly and can lead to spurious inferences. For example, if mosquitoes are reared in pans

(or kissing bugs in jars) to measure body size of emerging adults from different experimental conditions, the

lack of independence that arises from the aggregation

into pans (or jars) will inate the error value (a.k.a.

residual variance) of the LM, in some cases leading to

incorrect inferences when the LM is compared with a

model that explicitly models the lack of independence

because of the aggregation into a functional experimental unit (i.e., the pan or jar). More than 20 yr ago,

because of the limited statistical toolbox in ecology,

this issue was a major problem for the correct analysis

of datasets from studies with design constraints (Hurlbert 1984). However, strategies to handle the problem

of pseudoreplication were around at the time. For

example, in evolutionary ecology, individuals have

different degrees of common descent, and this variability by itself is often a subject of study. In the 1980s,

it was common to use nested half-sibling designs to

estimate the variance of families and individuals belonging to those families (Conner and Hartl 2004).

Also, the use of dened designs such as Greco-Roman

squares, Roman squares, and fractional factorials was

control (Montgomery 2005). Some of these balanced

designs were even used in studies of medically important insects (Carpenter 1982, Chesson 1984). In

fact, sophistication in randomization, when possible,

can be very useful to evaluate the impact of strategies

to control human-vector contact (Kirby et al. 2008).

For example, randomized control trials have been

used to demonstrate the importance of mosquito

screening in reducing the risk of malaria transmission

(Kirby et al. 2009). However, one of the major limitations of designs that handle pseudoreplication is the

need for balanced designs, i.e., an equal number of

replicates per treatment. The inability to analyze unbalanced designs with unequal number of replicates

per treatment has been overcome with the development of maximum likelihood methods, especially the

restricted maximum likelihood method and its application to estimate LMEM (Pinheiro and Bates 2000).

LMEM has the same fundamental assumptions of the

LM; it tries to explain the sources of variability across

a set of individual units of observation (yi) as function

of a series of n independent variables (x1, x2, . . . ., xn),

referred to as xed factors, but it can incorporate

additional sources of variability (the random factors),

besides the error (). The random factors can accommodate the lack of independence among the individual units of observation as a result of spatial, temporal,

genetic, or any exogenous environmental factor that is

not fully randomized. For example, the variance of

functional experimental units such as pans for mosquitoes or jars for kissing bugs can be explicitly modeled, thus allowing the proper estimation of the

error variance, and thus limiting the chances of

committing a type II error, i.e., rejecting the null

hypothesis when true. Therefore, LMEMs allow the

statistical analysis of pseudoreplicated data. The

next section will provide a series of examples illustrating the use of LMEMs and how they compare

with similar LMs. The data used in the examples and

code to perform the analyses using the open source

statistical software R are included as supplementary

online material (http://www.envs.emory.edu/research/

Chaves_SOM_Pseudoreplication.html).

LMs Versus LMEMs

Factorial Designs. To illustrate the most basic differences between LMs and LMEMs, I reference a eld

experiment designed to study oviposition by Culex

quinquefasciatus Say in Atlanta, GA (Chaves et al.

2009). Cx. quinquefasciatus larvae are normally absent

from lotic systems, such as rivers and creeks. However,

several cities have relic sewage treatment systems

where runoff water and sewage are combined in the

same system, and after large rainfall events the combined sewage efuent can overow into urban water

bodies (Chaves et al. 2009). In this experiment, the

effects of combined sewage overow water and nutrient addition on oviposition site selection by this

mosquito species were studied using 10 experimental

pools (water containers) at four sites in a forest patch.

May 2010

and nutrient addition on the ln of total number of egg rafts 1

oviposited over 5 d by Culex quinquefasciatus in Atlanta, GA

(Chaves et al. 2009)

Factor

df

Sum

Mean

square square

Water (1)

Nutrient (2)

Water nutrient (3)

Block (4)

Error ()

1

1

1

3

9

0.653

50.634

0.013

0.467

0.58

0.653

50.634

0.013

0.156

0.064

factor (Fig. 1). The equivalent to equation 1 for a

LMEM is:

y il 1x 1 2x 2 3x 1x 2 l il

F Value

Pr(F)

10.1401

786.0312

0.2055

2.4186

0.01111*

4.54E-10*

0.66105

0.13339

This design was balanced. Original data and R code for analysis are

in the supplementary online material (http://www.envs.emory.edu/

research/Chaves_SOM_Pseudoreplication.html).

*Statistically signicant (P 0.05).

For this example, data will be used from the experiment when egg rafts were removed daily. Data use has

been restricted to a randomly extracted subsample

from the original data (only four pools per site) to

have a dataset similar to that of a balanced design. In

this experiment, oviposition (y) was measured by

counting the total number of egg rafts oviposited over

5 d. The experiment has three independent variables

(i.e., n 3): 1) x1 water quality (with two levels:

combined sewage overow water and tap water as

control); 2) x2 nutrient addition (added or absent

as control); and 3) x3 sites (four in total). Only x1

and x2 are factors (each with two levels), because x3

is an independent variable considered to test the

block effects of forest site on oviposition. Because

all treatments were present at each site, this is a

randomized block 2 2 factorial design, which is

randomized because both factors were present in all

four sites (blocks in the model), and 2 2 factorial

because each factor has two levels. The goal of a

factorial experiment is to test whether the factors

interact, which can be expressed by the following

LM:

y il 1x 1 2x 2 3x 1x 2 4x 3l il [1]

where is the average value of the observations; 1,

2, and 4 quantify the impact of each independent

variable; 3 the interaction of water quality and nutrient addition; and is the error, which is assumed to

be normally distributed. The subscript l denotes block

(site within the forest patch), and i is for individual

pools within a block (containers in a forest site).

Therefore, yil is the total number of rafts from a

given pool and block (i.e., container in a site). Table

1 shows the results of the analysis of variance for the

data using the model presented in (1). The natural

logarithm transformation of ln (y 1) is done to

normalize the data and fulll model assumptions.

Table 1 shows that neither block nor the interaction

between water quality and nutrient addition was

signicant (P 0.05).

The inuence of water quality and nutrients on Cx.

quinquefasciatus oviposition can be reanalyzed using a

LMEM. By contrast with the LM analysis, in which

inferences are done over blocks, the LMEM assumes

that blocks are random samples from a larger pop-

293

[2]

where , the s, and y and have the same interpretation as in equation 1, and quanties the variability

across the blocks, which is assumed to be normally

distributed. The signicance of factors can be tested

using F tests, which work well when designs are balanced (equal number of samples per treatment and

block) and parameters can be estimated using maximum likelihood. More generally, signicance may be

tested using parametric bootstraps, for balanced or

unbalanced designs where parameters are estimated

by restricted maximum likelihood (Pinheiro and Bates

2000, Faraway 2006). Table 2 shows the results of an

analysis of deviance, with parameters of equation 2

estimated using restricted maximum likelihood and

inference based on 1000 replications of parametric

bootstrap (an analysis in which datasets are simulated

and the results of likelihood ratio tests for studied

factors are compared with those of the true data to

compute the signicance of factors). In this example,

inference about the impact of water quality and nutrient addition is qualitatively similar using LM or

LMEM. However, LMEM provides additional insight;

it indicates oviposition is nely grained. The variance

at the individual container level is larger than at the

block level (Table 2: 0.059 and 0.024), a

pattern also observed in the original study for the full

dataset (Chaves et al. 2009). The data can be observed

in Fig. 2.

Constrained Designs. One of the major limitations

of the LM is that it is not suited to analyze datasets with

constraints in randomization. For example, all replicates from a treatment should be present in all blocks.

This is a frequent limitation in eld studies in which

features of a given landscape cannot be altered, an

underlying motivation behind split-plot designs, in

which some treatments do not vary across blocks.

Split-plots are widely used in agriculture (Faraway

2006) and economic entomology (Blumberg et al.

1997, Haile et al. 2000, Oyediran et al. 2007). However,

constraints in randomization can also arise as a product of other trade-offs in experimentation or by the

nature of the questions asked. For example, the original design of Chaves et al. (2009) was unbalanced in

the sense that each block had an unequal number of

replicates for each one of the treatments. However, for

each block the amount of total nutrients and water

quality was constant; one of the questions was to

determine the grain of mosquito perception for oviposition choices. The largest variance was among the

individual oviposition containers, indicating a nely

grained perception, in contrast to a scenario of

coarsely grained mosquito perception, in which the

largest variance would be expected for the blocks or

sites. To illustrate the analysis of constrained designs,

the original data of Chaves et al. (2009) are sampled

in such a way that all experimental pools with nutrients added and combined sewage overow water

294

Fig. 1. Blocks: xed or random? The left panel shows the case for blocks as a xed factor in LM, in which the assumption

is that inferences are exclusive for the observed blocks (within squares). The right panel shows the case for blocks as a random

factor in LMEMs, in which the observed blocks (within squares) come from a larger population (i.e., blocks inside and outside

squares). (Online gure in color.)

came from the same block (Fig. 2), and therefore the

design becomes unbalanced (Fig. 3). Thus, the LM

from equation 1 cannot be employed to analyze the

data, but the LMEM from equation 2 is suitable for

such an analysis. Results are presented in Table 3 and

are similar to those presented in Table 2, showing a

decreased variability in the blocks and a larger error.

In summary, LMEMs can uncover the same variance

pattern in data under pseudoreplication, a major advantage over LMs.

Table 2. Analysis of deviance for the effects of water quality

and nutrient addition on the ln of total number of egg rafts 1

oviposited over 5 d by Culex quinquefasciatus in Atlanta, GA

(Chaves et al. 2009)

Fixed

df

Log likelihood

LRT

Water (1)

Nutrient (2)

Water nutrient (3)

1

1

1

7.241

31.089

4.304

5.874

53.569

8.612

0.009*

0.000*

0.664

Random

Blocks ()

Error ()

Spatial Variability

Organisms can be clustered in space, for example,

the larvae of mosquitoes can be associated with only

certain habitats where eggs are oviposited, thus making their abundance autocorrelated in space (Pitcairn

et al. 1994). Several statistical tools can accommodate

the lack of spatial independence in data from eld

studies (Fortin and Dale 2005), and they have been

widely used with insects of medical importance

(Koenraadt et al. 2007, 2008; Vazquez-Prokopec et al.

2008). However, their description is outside the scope

of this article. LMEM can also be used to consider

spatial variability. LMEM are especially suitable for

cases when spatial scales are nested. For example,

mosquito larval samples coming from containers in

several houses that belong to the same neighborhood

are hierarchically nested. Several studies have used

this approach in recent studies on medically important

insects (Harrington et al. 2008, Chaves et al. 2009,

Gurtler et al. 2009). The LMEM tting procedure is

similar to the one used next to consider the lack of

temporal independence in longitudinal studies.

0.024

0.059

This design was balanced. Original data and R code for analysis are

in the supplementary online material (http://www.envs.emory.edu/

research/Chaves_SOM_Pseudoreplication.html). LRT, likelihood ratio test.

*Statistically signicant (P 0.05).

Longitudinal Studies

The fact that observations are repeated through

time in the same place (or from the same organisms)

can lead to data that are not independent and are

autocorrelated in time. One approach to this problem

May 2010

295

and nutrient addition on the ln of total number of egg rafts 1

oviposited over 5 d by Culex quinquefasciatus in Atlanta, GA

(Chaves et al. 2009)

Fixed

df

Log likelihood

LRT

Water (1)

Nutrient (2)

Water nutrient (3)

1

1

1

10.45

29.03

7.711

5.477

42.638

0.0409

0.012*

0.000*

0.801

Random

Blocks ()

Error ()

5.0405e-14

0.125

This design was unbalanced (because of the sampling from the full

dataset). Original data and R code for analysis are in the supplementary online material (http://www.envs.emory.edu/research/Chaves_

SOM_Pseudoreplication.html). LRT, likelihood ratio test.

*Statistically signicant (P 0.05).

logarithm number of Cx. quinquefasciatus egg rafts 1: (A) in

the balanced (Tables 1 and 2) and (B) unbalanced (Table 3)

block designs to study the effects of water quality and nutrient

enrichment on oviposition. Tap indicates tap water and CSO

indicates combined sewage overow water. Y stands for nutrient addition, and N for no additional nutrients. Data extracted

from Chaves et al. (2009). Original data are available in the

supplementary online material (http://www.envs.emory.edu/

research/Chaves_SOM_Pseudoreplication.html).

2006) and time series analysis techniques (Shumway

and Stoffer 2000, Chaves and Pascual 2007). Time

series techniques have been used for longitudinal

studies of some vectors (Hayes and Downs 1980,

Strickman 1988, Feliciangeli and Rabinovich 1998,

Scott et al. 2000, Salomon et al. 2004). Alternatively,

LMEM can model the lack of temporal independence

as a random factor, which is one of the many methods

Modeling the lack of temporal independence will be

illustrated by examining data from a study on the

biting and resting behavior of anophelines using experimental huts in three villages of western Venezuela

(Rubio-Palis and Curtis 1992). Mosquitoes were collected during two nights per month and by catching

the landing mosquitoes on the legs of two catchers

between 1900 and 0700 hours, inside and outside experimental huts. Although several species were found,

only data for Anopheles nuneztovari Gabaldo n from

Guaquitas collected between August 1988 and October 1989 will be analyzed in this study (Fig. 4). In this

case, the response or dependent variable (y) is the

total number of landings for all huts, as presented in

the original study (Rubio-Palis and Curtis 1992). The

xed factors are as follows: 1) x1, the site with two

levels, inside and outside the hut; 2) x2, the landing time

with 12 levels corresponding to the hours between 1900

and 0700 hours; and 3) x3, the rainfall season with two

levels: dry (December-May) and wet (JuneNovember). The random factors consider the different scales of

temporal variability: 1) l, the year l; 2) kl, the month k

within a given year l; 3) jkl, the sampling day j within a

given month k and year l; and 4) ijkl, which is the error

i (error for an observation belonging to day j within a

given month k and year l). All random factors are assumed to be independent and normally distributed. The

model equation is as follows:

y ijkl 1x 1 2x 2 3x 3 4x 2x 3

l kl jkl ijkl [3]

all samples with added nutrients and combined sewage overow water, and other treatments have unequal number

across other blocks. (Online gure in color.)

observations; 1, 2, and 3 quantify the impact of each

independent variable on the number of landings; and

4 the interaction of season and landing time. Note

that choice of factors is dictated by the study objective:

quantication of the seasonal nocturnal biting pattern

outside and inside the experimental huts. Such quantication requires landing time, site, and season to be

treated as xed independent variables. The other temporal variables need to be random factors accounting

296

Fig. 4. Boxplots (median and quartiles) for the hourly number of Anopheles nuneztovari landings: (A) outside the house,

dry season; (B) inside the house, dry season; (C) outside the house, wet season; (D) inside the house, wet season. Data

extracted from Rubio-Palis and Curtis (1992). Original data are available in the supplementary online material (http://

www.envs.emory.edu/research/Chaves_SOM_Pseudoreplication.html).

for the lack of independence that arises from the repeated measurements through time. There was no variability because of the year of the observation ( 0; see

supplementary online material http://www.envs.emory.

edu/research/Chaves_SOM_Pseudoreplication.html),

and a simpler model, without a parameter for the annual

variability, was t, as follows:

(P 0.05). Daily observations for each month were

more homogeneous, having a lower variance than

those observations across months. For comparison

purposes, a LM was also t, as follows:

y i 1x 1 2x 2 3x 3 4x 2x 3

i . . .

y ijk 1x 1 2x 2 3x 3 4x 2x 3 k

jk ijk [4]

Results for this model are presented in Table 4. The

interaction between season and time and the main

Table 4. Analysis of deviance for the effects of site, landing

time, and season on Anopheles nuneztovari abundance in Guaquitas, Venezuela (Rubio-Palis and Curtis 1992)

Factor

df

Log likelihood

LRT

Site (1)

Landing time (2)

Season (3)

Landing time

season (4)

1

11

1

11

3550

3619

3548

3542

15.1

154.2

11.2

124.4

0.000*

0.073

0.000*

0.000*

Random

Month ()

Day ()

Error ()

751.46

182.68

1669.42

online material (http://www.envs.emory.edu/research/Chaves_

SOM_Pseudoreplication.html). LRT, likelihood ratio test.

*Statistically signicant (P 0.05).

[5]

Table 5 shows the results for the analysis with equation 5. All factors are signicant with this model (P

0.05). However, when compared with the model with

random effects, the main effect for landing time is not

signicant when the lack of independence in the data

is properly modeled with a LMEM (Table 4). These

analyses illustrate one of the problems of incorrectly

Table 5. Analysis of variance for the effects of site, landing

time, and season on Anopheles nuneztovari abundance in Guaquitas, Venezuela (Rubio-Palis and Curtis 1992)

Factor

df

Sum

square

Mean

square

F Value

Pr(F)

1

17,914 17,914

6.727

0.009704*

Site (1)

5.5483

1.55E-08*

Landing time (2) 11 162,525 14,775

1 286,446 286,446 107.5666 2.2e-16*

Season (3)

11

77,851

7077

2.6577

0.002432*

Landing time

season (4)

Error ()

671 178,6849

2663

Original data and R code for analysis are in the supplementary

online material (http://www.envs.emory.edu/research/Chaves_

SOM_Pseudoreplication.html).

*Statistically signicant (P 0.05).

May 2010

modeling the lack of independence across observations: the LM rejects a null hypothesis that is true

(type II error) by saying that landing time by itself is

signicant (Table 5), when in reality it is only significant when considered in conjunction with the season

(Table 4).

Pseudoreplication: an Issue of the Past

As shown in this forum, pseudoreplication no longer

is an issue preventing the statistical analysis of experiments and eld studies. Current statistical tools such

as LMEM can model the lack of independence in eld

observations. However, pseudoreplication will most

likely always be present in any ecological study, because of the complexity of working with living organisms that constrains full randomization or limits the

number of replicates. Although other objects of study,

like molecules or atoms, are numerous and widespread, samples of living organisms are comparatively

few and organisms always are evolutionary and ecologically related at some scale. Although this forum has

been focused on demystifying statistical concepts and

presents how to use LMEM models to address the lack

of independence in datasets, the ingenuity of statisticians is laudable because many other techniques outside the scope of this article have been developed over

recent years. A best example includes the extension of

LMEM to accommodate non-normal observations in

generalized LMEMs (Bolker et al. 2009). Other tools

that do not consider the individual variability of observations, but rather the average across all samples,

like the generalized estimating equations (Faraway

2006), can address the lack of independence in observations, and have been used in the study of medically important insects (Lindblade et al. 2000, Gurevitz et al. 2009). A third line of new computer-based

tools, including neural networks, trees (Olden et al.

2008), and random forests (Ruiz et al. 2010), does not

have assumptions on data independence, and has been

successfully used to study insects of public health

importance (Hu et al. 2006, Ruiz et al. 2010). Thus,

pseudoreplication should no longer be considered as

a major aw that impairs the statistical analysis of

experiments and eld studies. Independence constraints in the manipulation and observation of organisms are adequately handled by many available statistical tools, thus enabling valid inferences from

valuable entomological data.

Acknowledgments

I am thankful to Yasmin Rubio-Palis for sharing her original data on Anopheles nuneztovari from Guaquitas, Venezuela. This work was funded by a Gorgas Research Award from

the American Society of Tropical Medicine and Hygiene and

Emory University. This work also beneted from comments

by the editor, anonymous reviewers, Jorge Rabinovich,

Nicole Gottdenker, and Greg Decker, and helpful discussions

from a National Institutes of Health-Research and Policy on

Infectious Disease Dynamics (NIH-RAPIDD) study group

on mosquito-borne diseases.

297

References Cited

Alto, B. W., and S. A. Juliano. 2001a. Precipitation and temperature effects on populations of Aedes albopictus

(Diptera: Culicidae): implications for range expansion.

J. Med. Entomol. 38: 646 656.

Alto, B. W., and S. A. Juliano. 2001b. Temperature effects on

the dynamics of Aedes albopictus (Diptera: Culicidae)

populations in the laboratory. J. Med. Entomol. 38: 548

556.

Blumberg, A.J.Y., P. F. Hendrix, and D. A. Crossley. 1997.

Effects of nitrogen source on arthropod biomass in notillage and conventional tillage grain sorghum agroecosystems. Environ. Entomol. 26: 3137.

Bolker, B. M., M. E. Brooks, C. J. Clark, S. W. Geange, J. R.

Poulsen, M. H. Stevens, and J. S. White. 2009. Generalized linear mixed models: a practical guide for ecology

and evolution. Trends Ecol. Evol. 24: 127135.

Box, J. F. 1980. R. A. Fisher and the design of experiments,

19221926. Am. Stat. 34: 17.

Carpenter, S. R. 1982. Stemow chemistry: effects on population dynamics of detritivorous mosquitoes in tree-hole

ecosystems. Oecologia 53: 1 6.

Chaves, L. F., and M. Pascual. 2007. Comparing models for

early warning systems of neglected tropical diseases.

PLoS Negl. Trop. Dis. 1: e33.

Chaves, L. F., C. L. Keogh, G. M. Vazquez-Prokopec, and

U. D. Kitron. 2009. Combined sewage overow enhances oviposition of Culex quinquefasciatus (Diptera:

Culicidae) in urban areas. J. Med. Entomol. 46: 220 226.

Chesson, J. 1984. Effect of notonectids (Hemiptera, Notonectidae) on mosquitos (Diptera, Culicidae): predation or selective oviposition. Environ. Entomol. 13: 531

538.

Conner, J. K., and D. L. Hartl. 2004. A primer of ecological

genetics. Sinauer, Sunderland, MA.

Faraway, J. J. 2006. Extending the linear model with R: generalized linear, mixed effects and nonparametric regression models. CRC, Boca Raton, FL.

Feliciangeli, M. D., and J. Rabinovich. 1998. Abundance of

Lutzomyia ovallesi but not Lu-gomezi (Diptera: Psychodidae) correlated with cutaneous leishmaniasis incidence

in north-central Venezuela. Med. Vet. Entomol. 12: 121

131.

Fisher, R. A. 1935. The design of experiments. Oliver &

Boyd, Edinburgh, United Kingdom.

Fortin, M. J., and M.R.T. Dale. 2005. Spatial analysis: a guide

for ecologists. Cambridge University Press, Cambridge,

United Kingdom.

Gurevitz, J. M., U. Kitron, and R. E. Gurtler. 2009. Temporal

dynamics of ight muscle development in Triatoma infestans (Hemiptera: Reduviidae). J. Med. Entomol. 46:

10211024.

Gurtler, R. E., F. M. Garelli, and H. D. Coto. 2009. Effects

of a ve-year citywide intervention program to control

Aedes aegypti and prevent dengue outbreaks in northern

Argentina. PLoS Negl. Trop. Dis. 3: e427.

Haile, F. J., D. L. Kerns, J. M. Richardson, and L. G. Higley.

2000. Impact of insecticides and surfactant on lettuce

physiology and yield. J. Econ. Entomol. 93: 788 794.

Harrington, L. C., A. Ponlawat, J. D. Edman, T. W. Scott, and

F. Vermeylen. 2008. Inuence of container size, location, and time of day on oviposition patterns of the dengue vector, Aedes aegypti, in Thailand. Vector Borne Zoonotic Dis. 8: 415 423.

Hayes, J., and T. D. Downs. 1980. Seasonal changes in an

isolated population of Culex pipiens quinquefasciatus

(Dipter: Culicidae): a time series analysis. J. Med. Entomol. 17: 63 69.

298

Heffner, R. A., M. J. Butler, and C. K. Reilly. 1996. Pseudoreplication revisited. Ecology 77: 2558 2562.

Hu, W., S. Tong, K. Mengersen, B. Oldenburg, and P. Dale.

2006. Mosquito species (Diptera: Culicidae) and the

transmission of Ross River virus in Brisbane, Australia.

J. Med. Entomol. 43: 375381.

Hurlbert, S. H. 1984. Pseudoreplication and the design of

ecological eld experiments. Ecol. Monogr. 54: 187211.

Kirby, M., P. Milligan, D. Conway, and S. Lindsay. 2008.

Study protocol for a three-armed randomized controlled

trial to assess whether house screening can reduce exposure to malaria vectors and reduce malaria transmission

in The Gambia. Trials 9: 33.

Kirby, M. J., D. Ameh, C. Bottomley, C. Green, M. Jawara,

P. J. Milligan, P. C. Snell, D. J. Conway, and S. W. Lindsay.

2009. Effect of two different house screening interventions on exposure to malaria vectors and on anemia in

children in The Gambia: a randomized controlled trial.

Lancet 374: 998 1009.

Koenraadt, C.J.M., J. Aldstadt, U. Kijchalao, A. Kengluecha,

J. W. Jones, and T. W. Scott. 2007. Spatial and temporal

patterns in the recovery of Aedes aegypti (Diptera: Culicidae) populations after insecticide treatment. J. Med.

Entomol. 44: 6571.

Koenraadt, C.J.M., J. Aldstadt, U. Kijchalao, R. Sithiprasasna,

A. Getis, J. W. Jones, and T. W. Scott. 2008. Spatial and

temporal patterns in pupal and adult production of the

dengue vector Aedes aegypti in Kamphaeng Phet, Thailand. Am. J. Trop. Med. Hyg. 79: 230 238.

Lindblade, K. A., E. D. Walker, A. W. Onapa, J. Katungu, and

M. L. Wilson. 2000. Land use change alters malaria

transmission parameters by modifying temperature in a

highland area of Uganda. Trop. Med. Int. Health 5: 263

274.

Millar, R. B., and M. J. Anderson. 2004. Remedies for pseudoreplication. Fisheries Res. 70: 397 407.

Montgomery, D. C. 2005. Design and analysis of experiments. Wiley, New York, NY.

Oksanen, L. 2001. Logic of experiments in ecology: is pseudoreplication a pseudoissue? Oikos 94: 2738.

Olden, J. D., J. J. Lawler, and N. LeRoy-Poff. 2008. Machine

learning methods without tears: a primer for ecologists. Q.

Rev. Biol. 83: 171193.

Oyediran, I. O., M. L. Higdon, T. L. Clark, and B. E. Hibbard.

2007. Interactions of alternate hosts, postemergence

western corn rootworm (Coleoptera: Chrysomelidae)

damage and adult emergence. J. Econ. Entomol. 100:

557565.

Pinheiro, J. C., and D. M. Bates. 2000. Mixed effects models

in S and S-plus. Springer, New York, NY.

Pitcairn, M. J., L. T. Wilson, R. K. Washino, and E. Rejmankova. 1994. Spatial patterns of Anopheles freeborni and

Culex tarsalis (Diptera: Culicidae) larvae in California

rice elds. J. Med. Entomol. 31: 54553.

Reiskind, M. H., and M. L. Wilson. 2004. Culex restuans

(Diptera: Culicidae) oviposition behavior determined by

larval habitat quality and quantity in southeastern Michigan. J. Med. Entomol. 41: 179 86.

Rubio-Palis, Y., and C. F. Curtis. 1992. Biting and resting

behavior of anophelines in western Venezuela and implications for control of malaria transmission. Med. Vet.

Entomol. 6: 325334.

Ruiz, M. O., L. F. Chaves, G. L. Hamer, T. Sun, W. M. Brown,

E. D. Walker, L. Haramis, T. L. Goldberg, and U. D.

Kitron. 2010. Local impact of temperature and precipitation on West Nile virus in Culex species mosquitoes in

northeast Illinois, U.S.A. Parasit. Vectors 3: 19.

Salomon, O. D., M. L. Wilson, L. E. Munstermann, and B. L.

Travi. 2004. Spatial and temporal patterns of phlebotomine sand ies (Diptera: Psychodidae) in a cutaneous

leishmaniasis focus in northern Argentina. J. Med. Entomol. 41: 3339.

Scott, T. W., A. C. Morrison, L. H. Lorenz, G. G. Clark, D.

Strickman, P. Kittayapong, H. Zhou, and J. D. Edman.

2000. Longitudinal studies of Aedes aegypti (Diptera: Culicidae) in Thailand and Puerto Rico: population dynamics. J. Med. Entomol. 37: 77 88.

Shumway, R. H., and D. S. Stoffer. 2000. Time series analysis

and its applications. Springer, New York, NY.

Strickman, D. 1988. Rate of oviposition by Culex quinquefasciatus in San Antonio, Texas, during three years. J. Am.

Mosq. Control Assoc. 4: 339 344.

Vazquez-Prokopec, G. M., M. C. Cecere, U. Kitron, and R. E.

Gurtler. 2008. Environmental and demographic factors

determining the spatial distribution of Triatoma guasayana in peridomestic and semi-sylvatic habitats of rural

northwestern Argentina. Med. Vet. Entomol. 22: 273282.

Received 12 October 2009; accepted 20 January 2010.

- Kumar 2014Transféré parmerymistika afred
- An Example of a HypothesisTransféré parAjibade Taofik
- Business Statistics (A Decision Making Approach) ANOVATransféré parFarhad
- Verslycke Et Al. CEA and Scope Growth in Neomysis Integer Exposure to ChlorpyrifosTransféré parArmando Martinez
- 4. Edu Sci - Biographic - Samuel TuffourTransféré parTJPRC Publications
- 22 Bivariate StatsTransféré parmdsabbir
- 1-s2.0-S0261306911001701-mainTransféré parSergio Muñoz
- Lesson 11Transféré parnaveengargns
- Rm 9Transféré parAshish Aguiar
- Economic Analysis and Modelling of Effects of NPK Fertilizer Levels on Yield of YamTransféré parKabir Opeyemi Olorede
- j.2044-8279.1981.tb02471.xTransféré parMarina Louise
- fulltext.pdfTransféré parSaba Khan
- STATS 2 PRACTICAL ASSIGNMENT revised.docxTransféré parJanhavi Patel
- IJSRED-V2I1P25Transféré parIJSRED
- Bio Project V2Transféré parAnup Abhipsit
- MPRA Paper 29640Transféré parShahid Ali
- CSCC6201604V04S01A0002.pdfTransféré parNavin Bravo
- Hasil Anova One WayTransféré parardyanpradana007
- Rendimiento en FutbolTransféré parCarlos Raul Tamargo
- CBRM - Methodology used in Social Science and ManagementTransféré parGuru Prasad
- hw8Transféré parAlex Chen
- Misconceptions of Elementary School Students About Comparing Decimal NumbersTransféré parWSTee
- MBA133 Courseplan LTransféré parSoham Samanta
- 18. Agri Sci - Ijasr -Characterization of Drudgery in - MrunaliniTransféré parTJPRC Publications
- Chapter 8Transféré parSamuel
- Chap 3Transféré parShreshtha Verma
- Hypotheses Test 1 HandoutTransféré parTabbara Mohamed
- AP Microeconnomics Practice Exam 1Transféré parVarun Sharma
- 2017-4Transféré parErsel KURT
- 2011-04!06!142915 Revised Extra Credit Homework2Transféré parMikaela Sun

- riskTransféré parWilmer Orihuela
- Inquiry into the 216 Australian CensusTransféré parisraelrt
- Elasticsearch Essentials - Sample ChapterTransféré parPackt Publishing
- UT Dallas Syllabus for taught by John Wiorkowski (wiorkow)Transféré parUT Dallas Provost's Technology Group
- A Comparative Study of Centroid-Based and Naïve Bayes Classifiers for Document CategorizationTransféré parAnonymous 7VPPkWS8O
- Repeatability and Reproducability, explained by Alex LauTransféré parOsama Idrees
- MIT6_041F10_rec17.pdfTransféré parManikanta Reddy Manu
- Summary of Travel Trends 2001Transféré parnyc
- Total Ass Quiz 630Transféré parSyed Faisal Bukhari
- Package SurveyTransféré paroctavioascheri
- Introduction & Desrciptive Biostatistics by Jehanzeb KhanTransféré parJehanzebkhanyousafza
- Advanced Econometrics - 1985 - 1era Edición - AmemiyaTransféré parAdan Graus Rios
- ch8Transféré parSajid Rasool
- NBayesLogRegTransféré parhagik
- MTE_sol_2018Transféré parZahid Khan
- Estimate State-Space Models in System Identification Tool - MATLAB & SimulinkTransféré parJuan Manuel Mauro
- Netflix Stats Report January 2018Transféré parSantiago Fernandez Moreno
- Risk-manager.pdfTransféré parNickolay Lobanov
- Lingad Et Al FinalTransféré parLouie Angelo Navarro Pacho
- Writing for Sociology Guide Second EditionTransféré parMohammed Salih
- 12.Probstat_Ch12-Tes Hipotesis 2Transféré parmister
- 419-1317-1-PBTransféré parmaxmcm
- Confidence IntervalsTransféré parNeko Plus
- Citb Skills and Training in the Construction Industry_final Report 2014Transféré parAbid Shah
- STAT 2507_Final_2010FTransféré parexamkiller
- Multiple RegressionTransféré parNarayanan Subramanian
- Regression AnalysisTransféré parRHA
- BUILDING-SPECIFIC LOSS ESTIMATION METHODS(chp1-5).pdfTransféré parRomina Ilina-Posea
- CHE32M2 - Statistical Process Control - Tut on the 3 charts and Q-Test 22 Oct 2018.docxTransféré parsoyiso
- Time Series QuestionsTransféré parakriti_08

## Bien plus que des documents.

Découvrez tout ce que Scribd a à offrir, dont les livres et les livres audio des principaux éditeurs.

Annulez à tout moment.