Combining quantitative and qualitative

approaches
by Katrin Niglas
http://www.leeds.ac.uk/educol/documents/00001544.htm

katrin@tpu.ee
Tallinn Pedagogical University
Narva mnt 25, Tallinn, 10120, ESTONIA

Introduction

Last year in Lahti I gave a paper concerning the debates (sometimes called as
"paradigm wars") about differences and similarities between quantitative and
qualitative research. I also presented the results of my small-scale investigation,
which showed that there were studies, which combined qualitative and quantitative
approaches in different ways. In this paper I want to look further and address some
problems concerning the use and integration of multiple methods in a social
scientific study.

In a long run there are three different widely advocated positions towards the
possibility and usefulness to use quantitative and qualitative approaches in
complimentary, combined or mixed ways:

The advocates of the first position, which I would call strong paradigmatic view,
declare that only one of those approaches is good/appropriate/scientific enough for
the inquiry about the social life. They say that quantitative and qualitative research
methodologies are tightly bound to different mutually exclusive epistemological
positions. From here follows that there is no point even to talk about the possibility
of combining or mixing of those approaches. The proponents of this position are
sometimes called purists.

The advocates of the second position, which I would call week paradigmatic
view, are somewhat more tolerant towards different methodologies saying that both
of them can be used and are useful, but as they carry with them different
philosophical underpinnings they are suitable in very different situations and
contexts and therefore one can not and should not mix or combine quantitative and
qualitative approaches in the framework of one study. The proponents of this view
are sometimes called situationalists.
The advocates of the third position regard quantitative and qualitative approaches
both as useful and proper ways of going to study the social world. Although they
see some major differences between quantitative and qualitative research they also
see some important similarities between them and advocate the integrated use of
different methodologies if this can advance our understanding about the
phenomenon under the investigation. The proponents of this position are
sometimes called pragmatists.

All of these three positions bring up some skeptical questions and problems one
needs to address and solve. In this paper I will take the pragmatist position, which
means that I will not question the feasibility of combining quantitative and
qualitative ways of doing research in general. I rather

try to look more closely on problems, which we have to be aware of in the process
of doing so.

Thus, I will not discuss problems, which paradigmatic view brings with it as this
subject has been the focus of many previous papers (including mine from the last
conference in Lahti).

Calls for multimethod approach.

Although the calls for the use of multiple methods in the framework of one study
are maybe even older than the quantitative-qualitative debate, the area of 'how,
when and why different methods might be combined' has got much less attention
than the philosophical aspects of paradigmatic view (Bryman 1988, 155). One can
not say that there is a complete lack of literature concerning different aspects of
combining divergent methodologies. Still most of the literature, which classifies
under the broad area of combining qualitative and quantitative approaches, are
arguments why this integration is possible and needed. On the other hand there is a
considerable number of papers either describing authors' own experiences on
integrating some aspects of quantitative and qualitative methodologies or following
so called 'case law' approach where a number of different experiences are
assembled together and called upon as exemplars one could follow (see for
example Brannen 1992, Brown et al. 1996, Bryman 1988, Caracelli & Greene
1993, Carey 1993, Maxwell et al. 1986).

As an example of the early call for leaving our methodological preconceptions

behind us and for considering all possible ways for advancing our knowledge
about the important aspects of social life I would like to quote Trow's paper where
he suggested that we, researchers in social sciences, should:

'get on with the business of attacking our problems with the widest array of conceptual and
methodological tools that we possess and they demand' (Trow 1957, 35; also quoted in
Brewer & Hunter 1989).
Approximately as early Donald Campbell and his co-authors published several
papers where they advocated the use of multitrait-multimethod matrixes and
triangulation of measurement for validation, proposed 'transition experiments' and
quasi-experimental designs (see Campbell 1957, Campbell & Fiske 1959,
Campbell & Stanley 1963, Webb et al. 1966). Although remaining in the
framework of quantitative tradition we can see in these early works the attempt to
advocate the use of multiple methods as well as the possibility to mix some aspects
of quantitative and qualitative methodologies.

Triangulation

Drawing on these ideas Denzin (1978) developed the concept of triangulation - the
term that is probably most widely used to denote any attempt to combine or mix
different methods in a research study. As it often happens, the most widely used
terms tend to be the most overused and abused terms as well, and 'triangulation' is
not an exception here I think. One could draw obvious parallels in how the term's
'paradigm' and 'triangulation' have lost their initial quite narrow and well-defined
meaning and became to denote something general and indefinite.

However, by Denzin triangulation means more than using multiple measurements

of the same phenomenon - in addition to the use of diverse data, it involves
combining different methods and theories, as well as perspectives of different
investigators. Denzin (1978) has clearly identified four different types of
triangulation:

data triangulation - the use of variety of data sources and data sets in a study. Data
may be both qualitative and quantitative, gathered by different methods or by the
same method from different sources or at different times.

investigator triangulation - the use of several different researchers. Here the

importance of partnership and teamwork is underlined as the way of bringing in
different perspectives.

theory triangulation - the use of different theoretical viewpoints for determining

competing hypotheses as well as for interpreting the single set of data.

methodological triangulation - the use of multiple methods to study a single

problem or phenomenon. It may also include the use of the same method on
different occasions and situations.

We can see that the concept of triangulation is based on the assumption that by
using several data sources, methods and investigators one can neutralize bias
inherent in one particular data source, investigator or method (Jick 1979). It is
often stressed out that different methods have different weaknesses and strengths
and therefore the main effect triangulation can offer is to overcome the weaknesses
of any single method. Thus, if we use several different methods for investigation of
the phenomenon of our interest and the results provide mutual confirmation we can
be more sure that our results are valid. Within this context, quantitative and
qualitative approaches are usually seen as different ways of studying the same
phenomenon and able to answer the same research questions (Bryman 1988).

Although the perspective of triangulation seems to be very promising several

authors have warned us about the hidden problems that the combined use of
qualitative and quantitative methods for the purposes of triangulation can bring
with it. Bryman (1992) has raised three alarming questions. First, as quantitative
and qualitative research have different preoccupations it is highly questionable
whether they are tapping the same things even when they are examining apparently
similar issues. Second, if quantitative and qualitative findings do not confirm each
other how should the researcher respond. And third, if the conflict in results is
present what it actually means and comprises. Thus, in the context of combining
qualitative and quantitative approaches the concept of triangulation is not as
unproblematic as it may appear.

On the other hand in the wider framework of integrated use of qualitative and
quantitative approaches the triangulation is offering quite limited possibilities. As
in the case of triangulation the results of different methods are supposed to validate
each other it means that different methods have to be highly independent
throughout the study. This approach excludes the possibility to mix quantitative
and qualitative aspects on different levels of investigation. For example one of the
few books devoted entirely to the problems of combining quantitative and
qualitative approaches Multimethod Research. A Synthesis of Styles by Brewer and
Hunter (1989) is largely constrained by the framework of triangulation although
the authors mention the other possibilities for integration as well. They classify
studies into three categories: a) monomethod studies, b) composite method studies,
which combine some elements of the basic monomethod styles and c) multimethod
studies, which combine the basic styles of research. The authors give their clear
preference to the multimethod designs because according to them composite
methods 'comprise some of the basic methods' sources of sterngth' and they 'fail to
provide the opportunity for triangulated measurement and hypothesis testing, and
the protection against monomethod bias, that the multimethod strategy provides'
(Brewer and Hunter 1989, 81).

Other rationales for combining quantitative and qualitative research

Regardless of this extensive critique of composite method designs several studies

have indicated that we can find considerable number of studies which actually
combine some elements of quantitative and qualitative approaches on the various
stages of the study (See for example Bryman 1988, Datta 1994, Greene et al. 1989,
Niglas 1999). Maybe the most interesting (although definitely not the most
systematic) of theses studies is Datta's analysis of several papers given to her as
examples of good qualitative and quantitative research by the proponents of
monomethod approach. Her conclusion was that 'the best examples of both
paradigms seem actually to be mixed models' (Datta 1994: 67). As a result of the
extensive literature review and a small-scale empirical analysis of published
research papers I have suggested that quantitative and qualitative approaches have
been combined in various ways and various levels of the inquiry. Different
possibilities for mixing quantitative and qualitative research can be illustrated by
the Figure 1 (Niglas 1999). Thus, we can see that in the practice researchers mix
and combine qualitative and quantitative methodologies, but the question remains
if this kind of action has to be approved and what is the rationale for doing so.

Figure 1. Different levels of research in practice

There are some methodologists who propose that the combination of various
elements of quantitative and qualitative approaches can offer much wider
possibilities than Brewer and Hunter describe (See for example Brannen 1992,
Bryman 1992, Datta 1994, Patton 1990, Cresswell 1995, Tashakkori and Teddlie
1998). Although these authors see the potential of studies with various mixed
designs, their overall concern seems to be that we still lack a comprehensive
theoretical background for mixed-model designs and therefore mixed-model
studies can seem to many practitioners, consumers and evaluators of the research
as 'mixed-up models' one can not rely on (Datta 1994, 59).
There have been several attempts to clarify the issue and develop taxonomies for
classification of studies combining quantitative and qualitative approaches. For
example Mark and Shotland (1987) introduce in addition to triangulation two other
ways of combining methods to enhance an investigation. They describe
the bracketing model, which says that we should consider the results of different
methods as alternative estimates. Thus, by using methods that are biased in
opposite directions the true value can be bracketed. The third model they describe
is complementary multiplism. Here diverse methods play complementary roles and
offer different viewpoints; together, they provide evidence that is markedly
strengthened.

In one of the most important books on the field Bryman (1988) described in
addition to triangulation ten other ways in which quantitative and qualitative
research have been combined in the research practice. Although it is a quite long
list Bryman conceded that very likely it is not exhaustive (See Table 1). Drawing
on the results of extensive literature review Greene, Caracelli and Graham (1989)
developed a system consisting of five different mixed-method purposes. The
results of their empirical study proved this taxonomy to be exhaustive as it was
possible to match all mixed-method studies, they looked at, to one or more of these
five purposes. I have tried to compare these two lists of purposes for mixed-
method studies in Table 1. We can see that there are some remarkable similarities:
in addition to triangulation the common purposes seem to be development and
expansion. In the former case the results from qualitative research help to inform
the use of quantitative research or vice versa and in the latter the quantitative and
qualitative approaches are used sequentially on different stages of the study. It is
not so easy to draw parallels between other categories as many of Bryman's ways
of integration can serve diverse purposes from complementarity through initiation
to expansion.

Table 1: Purposes for combining quantitative and qualitative approaches

Source: Bryman 1988 Source: Greene et al. 1989
Triangulation Triangulation - seeking convergence of results
from different methods.
Qualitative research facilitates quantitative
research
Quantitative research facilitates qualitative
research
Qualitative research facilitates the interpretation of Complementarity - clarification, illustration,
relationships between variables interpretation of the results from one method with
the results from the other

Quantitative research captures the structure and

qualitative research the process
Combining qualitative and quantitative research
helps to bridge the gulf between 'macro' and 'micro'
levels
Quantitative research helps add generalizability
Quantitative and qualitative research are used on
different stages of a longitudinal study

Hybrids which have elements of both research

traditions

In regard to the previous discussion about mixing various elements of quantitative

and qualitative research it is interesting to notice that the classification given by
Greene et al. does not distinguish studies with mixed design elements form other
methodological mixes. Nevertheless in their study the authors use some other
variables which allow to speculate that some of the papers studied belonged to the
former class. On the other hand Bryman has defined separate category for studies
which have design elements of both research traditions. This category remains
relatively weakly described by the author, although in his later work he discusses
some aspects of these 'hybrid' designs (Bryman 1992).

Taxonomies for studies combining quantitative and qualitative approach.

Looking for further references to mixed designs we can see that as early as 1980
Patton added a chapter on 'methodological mixes' to his widely used textbook
about qualitative evaluation methods. While also accepting the importance of
triangulation he wrote that the second way of 'achieving methodological
heterogeneity ... is to borrow and combine parts from pure methodological
strategies, thus creating mixed methodological strategies' (Patton 1980, 109, italics
added). Similarly Cresswell describes in his textbook three different designs for
combined use of quantitative and qualitative research: two-phase design,
dominant-less dominant design and mixed-methodology design, where the
researcher 'would mix aspects of qualitative and quantitative paradigm at all or
many methodological steps in the design' (Cresswell 1995, 178).

What we can learn from these different sources is that, as on any developing area,
there is a lack of terminological and even conceptual clarity and coherence: we can
find many different labels for the same ideas; at the same time authors use same
terms in different meanings (See Table 2)., Recently Abbas Tashakkori and
Charles Teddlie (1998) have made an attempt to develop exhaustive taxonomies
for studies combining quantitative and qualitative approaches in different ways.
They have not chosen to proceed from different purposes that combination of
different methods can serve but from the way in which different methods are
used/combined in a particular study. Similarly to Brewer and Hunter (1989) they
divide all studies into three main types: monomethod studies, mixed method
studies, and mixed model studies. Each of these basic types of studies is further
divided into subcategories. The basic types are defined as follows:
Monomethod studies follow in all stages of the inquiry 'one of the predominant
paradigms' (Tashakkori & Teddlie 1998: 17). Thus, they are either purely
quantitative or purely qualitative studies.

Mixed method studies combine the quantitative and qualitative approaches in a

single study or multiphased study. Here quantitative and qualitative approaches
seem regarded relatively independent, as the authors stress that all mixed method
designs use triangulation techniques.

Mixed model studies 'combine the qualitative and quantitative approaches within
different phases of the research process' (Op sit: 19). Here qualitative and
quantitative methodologies can be interwoven in different ways. They can be
present as the single application on different stages of the study or they may be
used simultaneously in integrated manner on the same pahse(s) of the inquiry.

In the Table 2 I have tried to compare classifications of studies that different

authors have proposed. Studying their definitions of given categories it becomes
clear that in spite of different labels there are considerable similarities between
classifications. Although there is no one-to-one correspondents it seems feasible to
organize classifications into three columns so that categories in each column are
conceptually close to each other. Thus, I think that the broad classification of
studies into three categories is useful and that definitions given by Tashakkori and
Teddlie (1998) are conceptually reasonable. What seems to me a little questionable
is their choice of labels for these categories. First, the use of the term 'mixed' for
the second category gives a somewhat different depiction as the definition itself. It
was made quite clear that in this category while combined in one study the
quantitative and qualitative approaches remain relatively independent at least until
the interpretation phase. The word 'mix' on the other hand usually carries an
opposite notion. For example the first meaning that The Oxford Dictionary of
Current English gives for the term 'mix' is:

'combine or put together (two or more substances or things) so that the

constituents of each are diffused among those of the other(s)' (Thompson
1996: 571).

Therefore it seems to me that the use of the word 'multi' or 'multiple' would be
more appropriate for this category.

The second problematic word is 'method' as in my view it is more often connected

with concrete techniques for data gathering and data analysis than with the whole
set of methodological issues. Therefore the label 'monomethod studies' sounds too
restrictive, as studies in the first category can exploit in the framework of one
methodological approach (either quantitative or qualitative) several data sources
and data gathering instruments. But it is not so easy to replace that term as
conceptually better terms are considerably longer and inconvenient to use.
Therefore, in the following, I will continue to use the term 'method' in its wider
sense, comprising all stages of the research.

Finally, as both terms 'multimethod' and 'mixed-method' have been used as an

umbrella-terms for all possible designs, where any combination of quantitative and
qualitative approaches occurs, the more restricted use of these terms may be
confusing, but I could not find a way to substitute them. Thus, in the first row of
Table 2 I have given labels and short definitions for the three broad categories of
research designs.

Table 2: Proposed classifications of studies by their ways of using/combining

quantitative and qualitative approaches
Proposed classification: pure designs: multi-method designs: mixed designs:

purely quantitative or designs where both designs where elements of

Tashakkori & Teddlie
1998
Brewer & Hunter 1989
Cresswell 1995
Marks and Shotland 1987
purely qualitative designs quantitative and quantitative and
(may involve the use of qualitative approaches are qualitative approach are
several data sources used, but they remain combined in various ways
and/or data gathering relatively independent within different phases of
instruments from the same until the interpretation the study.
approach). stage.
monomethod studies mixed method studies mixed model studies
monomethod studies multimethod studies composite method studies
quantitative study; two-phase design; mixed-methodology design
dominant-less dominant
qualitative study design
quantitative study; triangulation;

qualitative study bracketing model;

* complementary multiplism
Bryman 1988 quantitative study; ten different ways of methodological hybrids
integration
qualitative study
Patton 1980 quantitative study; triangulation mixed-methodology design

qualitative study

*
* triangulation and bracketing model can be used within the purely quantitative or qualitative studies as well.

For better and more detailed descriptions of different possibilities of combining

quantitative and qualitative approaches this broad classification needs to be divided
into subcategories. For the first two categories this can be done quite easily, as
there are several methodological texts concerning the issue and we can see
considerable agreement between different authors. There are two important
dimensions, which can be the basis for the classification: timing and the
importance given to divergent methods in the study. If different methods are used
independently on different phases of the inquiry then we can talk about sequential
design; if they are used in a parallel way at the same time we can talk
about parallel/simultaneous design. Similarly, if different approaches are given
approximately the same weight in the study we can talk about equivalent status
design and if one method is clearly prevailing we can talk about dominant-less
dominant design. By combining these possibilities for the studies either using
several different methods within the same approach or for studies using both
quantitative and qualitative approaches we will get six subgroups for pure designs
and eight subgroups for multimethod designs (For full list of these subcategories
see Tashakkori & Teddlie 1998: 15).

The further classification of studies with mixed design is considerably more

complicated, as there are many different ways one can mix quantitative and
qualitative approaches within the framework of one study. As here the elements of
divergent approaches can occur on different phases of the inquiry it is essential for
any classification that we divide the empirical study into methodological stages. It
is common to include in methodological textbooks figures or tables describing
basic steps of the research process. For my own teaching purposes I have
developed the model given on the Figure 2. For the present discussion it is
important to notice that in my view decisions made on earlier steps influence the
decisions one can take on the later steps but there is no one-to-one relationship
between methods available on different stages. This means that there is a
possibility to combine quantitative and qualitative elements on any of described
stages of the research.

Figure 2. Methodological decisions to be made and steps to be taken in the

process of the empirical research study:
However, for the purposes of classification of studies with mixed designs less
detailed division may be more suitable, as the combination of six or even more
levels will produce too many subgroups. In their inquiry of mixed-method studies
Greene et al. (1989) identified two broad levels of methodology: paradigms and
methods, where method refers to data gathering and handling techniques
and paradigm refers to the wider set of methodological and philosophical ideas.
Patton (1980) operated with three levels and divided possibilities at all levels into
two basic types, one of them regarded as qualitative and the other as quantitative:
design (experimental v naturalistic), measurement (quantitative v qualitative) and
analysis (statistical v content/qualitative). By combining these two different
possibilities at all three levels he got two pure strategies and four mixed forms:

Mixed form I: QUAN  QUAL  QUAL

experimental design, qualitative measurement, content (qualitative) analysis

Mixed form II: QUAN  QUAL  QUAN

experimental design, qualitative measurement, and statistical analysis

Mixed form III: QUAL  QUAL  QUAN

naturalistic inquiry, qualitative measurement, statistical analysis

Mixed form IV: QUAL  QUAN  QUAN

naturalistic inquiry, quantitative measurement, statistical analysis

Tashakkori and Teddlie (1998) took over Patton's three-dimensional model

proposing some terminological changes and essential extensions. They gave
following labels to the three levels of research study: 1) type of inquiry
(exploratory investigations and confirmatory investigations), 2) data collection and
operations (qualitative and quantitative), 3) data analysis and inference (qualitative
and statistical). To get an exhaustive taxonomy Tashakkori and Teddlie added two
'rare' mixed forms to their classification, which Patton (1980) had excluded:

Mixed form V: QUAN  QUAN  QUAL

confirmatory investigation, quantitative data/operations, qualitative

analysis/inference

Mixed form VI: QUAL  QUAN  QUAL

exploratory investigation, quantitative data/operations, qualitative

analysis/inference

In addition to these six mixed types where on every stage of the study there is used
only single approach Tashakkori and Teddlie gave examples of studies with more
complex mixed designs where both quantitative and qualitative approach are used
one the same stage of the inquiry.

The authors claim that proposed three expanded dimensions with relevant
categories are representative of basic steps one has to take in the course of the
research study. However, by studying their definitions I see an important gap in
these dimensions. The first dimension type of investigation is based on the
'distinction between studies with a priori hypotheses (confirmatory investigation)
and those without a priori hypotheses (exploratory investigations)' (Tashakkori &
Teddlie 1998: 53). The second dimension data collection and operations concerns
in the first place 'the form of data', but it also includes the measurement techniques,
methods for establishing the reliability and validity of result as well as sampling
procedures (op sit: 54). My main concern here is that they exclude the overall
design or strategy from their dimensions or put it implicitly into the same
dimension with data collection techniques. I even tend to object the idea that
sampling should be in the same group with data collection techniques, as the
results of my small-scale study showed that there were studies with qualitative
design and sampling, which used mainly quantitative data collection techniques
(Niglas 1999). Thus, I propose that overall design and sampling techniques should
form the separate dimension between the type of the study and the data collection.
If we look at Patton's original classification we see that his first dimension
comprises the overall design of the study, but the weakness of his classification is
that the given categories: naturalistic inquiry versus experimental design are too
restrictive, excluding many design options there are available.

Discussion

In this paper I have given a short overview of the evolution of ideas how
quantitative and qualitative approaches can be combined in the social research to
advance our knowledge about important aspects of our life. We have seen that the
idea to use multiple methods in the framework of one study was proposed already
in the middle of the past century by influential methodologists like Campbell,
Stanley and others. Soon the idea was taken further suggesting that the
combination of quantitative and qualitative research, which were by many
methodologists seen as incommensurable opposites, is not only feasible and
beneficial in solving our puzzles, but can solve some problems the 'pure designs'
can not overcome. Relying mainly on the examples from the research practice
different authors have listed various reasons for combination of quantitative and
qualitative aspects in a single study. More recently the attempts to chart the area by
developing taxonomies for studies combining quantitative and qualitative research
in different ways have been made. In this paper I have described several
possibilities of this classification proposed by different authors and pointed to
some problems related to the definition of dimensions which could be the basis for
classification of mixed designs.

In this short review article it has not been possible to focus on full list of important
issues related with possibilities and ways of combining quantitative and qualitative
research. In spite of their rejection of the incompatibility thesis several authors
have pointed to important problems one has to be aware of and deal with while
combining different elements of quantitative and qualitative approaches (See for
example Bryman 1992). Further, there are strong arguments that it is
oversimplified and even wrong to talk about two clearly distinguishable
approaches as the differences within quantitative and qualitative approaches are not
smaller or less important than between these two categories (Hammersley 1995 a).
On the one hand this argument supports the idea that indeed we can and we do
combine or mix design elements usually connected with divergent approaches, but
on the other hand it questions the basis for taxonomies of mixed designs as
whatever dimensions we take it is not possible to define two clearly separate
categories: quantitative and qualitative. As an example I would mention the
confusion about the methods of manifest content analysis, which have been seen
by some authors as quantitative and by others as qualitative data analysis
techniques.
In summary it has to be said that although some aspects of combined use of
quantitative and qualitative approaches have been studied in more detail (see for
example the work on data analysis strategies for mixed-method studies by
Caracelli and Greene (1993)), there is a need for further research to clarify the
implications of various mixed designs to the research results.

References

Brannen, J. (1992) Combining qualitative and quantitative approaches: an

overview. In Brannen, J. (ed) Mixing Methods: Qualitative and Quantitative
Research. Aldershot: Avebury.

Brewer, J. & Hunter, A. (1989) Multimethod research: A synthesis of styles.

Newbury Park, CA: Sage.

Brown, S., Riddell, S. & Duffield, J. (1996). Possibilities and problems of small-
scale studies to unpack the findings of large-scale studies of school effectiveness.
In Gray, J., Reynolds, D., Fitz-Gibon, C. & Jesson, D. (eds) Merging traditions:
The future of research on school effectiveness and school improvement. London:
Cassell.

Bryman, A. (1988) Quantity and Quality in Social Research. London: Unwin

Hyman.

Bryman, A. (1992) Quantitative and qualitative research: further reflections on

their integration. In Brannen, J. (ed) Mixing Methods: Qualitative and Quantitative
Research. Aldershot: Avebury.

Campbell, D.T. & Fiske, D.W. (1959) Convergent and discriminant validation by
the multitrait-multimethod matrix. Psychological Bulletin, 65, pp. 81-105.

Campbell, D.T. & Stanley, J.C. (1959) Experimental and quasi-experimental

designs for research. Boston: Houghton Mifflin.

Campbell, D.T. (1957) Factors relevant to the validity of experiments in social

settings. Psychological Bulletin, 54(4), pp. 297-312.

Caracelli, V.W. & Greene, J.C. (1993) Data analysis strategies for mixed-method
evaluation designs. Educational Evaluation and Policy Analysis, 15(2), pp. 195-
207.

Carey, J.W. (1993) Linking qualitative and quantitative methods: Integrating

cultural factors into public health. Qualitative Health Research, 3, pp. 298-318.

Cresswell, J.W. (1995) Research design: Qualitative and quantitative

approaches. Thousand Oaks, CA:Sage.
Datta, L. (1994) Paradigm wars: A basis for peaceful coexistence and beyond. In
Reichardt, C.S. & Rallis, S.F. (eds) The qualitative-quantitative debate: New
perspectives. San Francisco: Jossey-Bass.

Denzin, N.K. (1978) The research act: An introduction to sociological methods.

New York: McGraw-Hill.

Greene, J.C., Caracelli, V.J. & Graham, W.F. (1989) Toward a conceptual
framework for mixed-method evaluation designs. Educational Evaluation and
Policy Analysis, 11, pp. 255-274.

Hammersley, M. (1992a) The Paradigm Wars: Reports From the Front. British
Journal of Sociology of Education, 13(1), pp.131-143.

Hammersley, M. (1992b) What's Wrong with Ethnography? Methodological

Explorations. London: Routlege.

Hammersley, M. (1995) Opening Up the Quantitative-qualitative

Divide. Education Section Review, 19(1), pp.2-15.

Howe, K. (1988) Against the Quantitative-qualitative Incompatibility Thesis or

Dogmas die Hard. Educational Researcher, 17(8), pp.10-16.

Jick, T.D. (1979) Mixing qualitative and quantitative methods: Triangulation in

action. Administrative Science Quarterly, 24, pp. 602-611.

Mark, M.M. & Shotland, R.L. (1987) Alternative models for the use of multiple
methods. In Mark, M.M. & Shotland, R.L. (eds) Multiple methods in program
evaluation: New Directions for Program Evaluation 35, pp. 95-100. San
Francisco: Jossey-Bass.

Maxwell, J.A., Bashook, P.G. & Sandlow, C.J. (1986) Combining ethnographic
and experimental methods in educational evaluation. In Fetterman, D.M. & Pitman
M.A. (eds), Educational evaluation: Ethnography in theory, practice, and politics.
Bewerly Hills, CA: Sage.

Niglas, K. (1999) Quantitative and Qualitative Inquiry in Educational Research: is

there a paradigmatic difference between them? Paper given at ECER 1999 in
Lahti. Paper is available electronically in Education-line:
http://www.leeds.ac.uk/educol/

Patton, M.Q. (1980) Qualitative evaluation and research methods. Newbury Park,
CA:Sage.
Reichardt, C.S. & Cook, T.D. (1979) Beyond Qualitative versus Quantitative
Methods. In Cook, T.D. & Reichardt, C.S. (eds) Qualitative and Quantitative
Methods in Evaluation Research. Beverly Hills, Calif.: Sage, pp. 7-32.

Reichardt, C.S. & Rallis, S.F. (1994) Qualitative and quantitative inquiries are not
incompatible: A call for a new partnership. In Reichardt, C.S. & Rallis, S.F.
(eds) The qualitative-quantitative debate: New perspectives. San Francisco:
Jossey-Bass.

Tashakkori, A. & Teddlie, C. (1998) Mixed Methodology: Combining Qualitative

and Quantitative Approaches. Thousand Oaks, CA:Sage.

Thompson, D. (ed) (1996) The Oxford Dictionary of Current English (revised 2nd
ed). Oxford University Press.

Trow, M. (1957) Comment on participant observation and interviewing: A

comparison. Human Organization, 16, pp. 33-35.

Webb, E.J., Campbell, D.T. Schwartz, R.D. & Sechrest, L. (1966) Unrobustive
measures.: Nonreactive research in the social sciences. Chicago: Rand McNally.

This document was added to the Education-line database on 05 October 2000

 Triangulation
Synonyms:

Confirming, Reinforcing, Rejecting

Triangulation facilitates validation of data through cross verification from

more than two sources. It tests the consistency of findings obtained
through different instruments and increases the chance to control, or at
least assess, some of the threats or multiple causes influencing our results.

Triangulation is not just about validation but about deepening and

widening one’s understanding. It can be used to produce innovation in
conceptual framing. It can lead to multi-perspective meta-
interpretations.[Triangulation is an] attempt to map out, or explain more
fully, the richness and complexity of human behavior by studying it from
more than one standpoint? - Cohen and Manion

Types

Denzin (1973, p.301) proposes four basic types of triangulation:

 Data triangulation: involves time, space, and persons

 Investigator triangulation: involves multiple researchers in an
investigation
 Theory triangulation: involves using more than one theoretical scheme in
the interpretation of the phenomenon
 Methodological triangulation: involves using more than one option to
gather data, such as interviews, observations, questionnaires, and
documents.

Reasons for triangulation

Carvalho and White (1997) propose four reasons for undertaking

triangulation:

 Enriching: The outputs of different informal and formal instruments add

value to each other by explaining different aspects of an issue
 Refuting: Where one set of options disproves a hypothesis generated by
another set of options.
 Confirming: Where one set of options confirms a hypothesis generated by
another set of options
 Explaining: Where one set of options sheds light on unexpected findings
derived from another set of options.

Triangulation to minimize bias

The problem with relying on just one option is to do with bias. There are
several types of bias encountered in research, and triangulation can help
with most of them.

 Measurement bias – Measurement bias is caused by the way in which you

collect data. Triangulation allows you to combine individual and group
research options to help reduce bias such as peer pressure on focus
group participants.
 Sampling bias – Sampling bias is when you don’t cover all of the
population you’re studying (omission bias) or you cover only some parts
because it’s more convenient (inclusion bias). Triangulation combines the
different strengths of these options to ensure you getting sufficient
coverage.
 Procedural bias – Procedural bias occurs when participants are put under
some kind of pressure to provide information. For example, doing “vox
pop” style interrupt polls might catch the participants unaware and thus
affect their answers. Triangulation allows us to combine short
engagements with longer engagements where participants have more
time to give considered responses.

Sources
Carvalho, S. and White, H. (1997). Combining the quantitative and
qualitative approaches to poverty measurement and analysis: The practice
and the potential. World Bank Technical Paper 366. Washington, D.C.:
World Bank

Denzin, Norman K. (1973). The research act: A theoretical introduction

to sociological methods. New Jersey: Transaction Publishers.
Kennedy, Patrick. (2009). How to combine multiple research options:
Practical Triangulation (http://johnnyholland.org/2009/08/20/practical-
triangulation)

Updated: 19th November 2018 - 12:15pm

https://www.betterevaluation.org/en/evaluation-options/triangulation