Vous êtes sur la page 1sur 15

How to Design a Mixed

Methods Study

by John W. Creswell, Ph.D. and Vicki L. Plano Clark, M.S.


University of Nebraska-Lincoln

Andrews University, July, 2004

How would you combine two


types of data? Objectives of the workshop:
Qualitative Quantitative ! Let’s design a mixed methods study
Text Data Numeric Data
! Let’s study how people learn mixed
methods research in this room?
This is a sample of a 2342543112232132
text file of words that 23322543
(or you can work on your own project
might be collected on
transcripts through 3122432432132433 and follow along at each step)
interviews, fieldnotes 32334441
from observations, or ! Let’s start with a title. Write a title.
from optically-scanned 2222111432143213
documents. 22111555 ! What data will we collect?
2331432432132433
32135432 3 4
What are types of quantitative and
qualitative data?

! Quantitative data ! Qualitative data ! Let’s identify our quantitative and


! Close-ended scales ! Open-ended
qualitative data collection
! Attitudinal/behavioral responses
scales ! Semi-structured

! Behavioral checklists interviews


! Census, attendance ! Semi-structured

records observations
! Records/documents

! Videotapes

5 6

Now let’s consider some reasons for


why we are collecting (and mixing)
both forms of data
Together quantitative and qualitative data provide both precise
!
measurement and generalizability of quantitative research and ! Let’s identify our reason for mixing
the in-depth, complex picture of qualitative research
! To validate quantitative results with qualitative data
! We do not have an adequate instrument. Thus, we need to
explore views and develop an instrument
! Our quantitative data provide a general explanation and we
need to follow-up with participants and have them explain the
quantitative results
! In our experiment, outcomes to be measured are not enough;
they need to be complemented by understanding the process of
participants

7 8
Requirement #1: Now let’s consider
whether we have the skills, time, and
So… resources?

! There are good reasons for gathering ! We need minimum skills in both
both forms of data qualitative and quantitative data
! But…there are certain requirements for collection. What do we need?
this to work best ! We need time and resources for
extensive data collection and analysis.
How much time and resources do we
need?

9 10

Requirement #2: The


audience(s)
! Write down the skills, time, and ! Does our audience appreciate both
resources we will need numbers and stories?
! Are they familiar with this design?
! Do they need to be educated?
! Are examples of published studies
available in our content area?

11 12
But audiences may not recognize
it yet because it is so new
! Let’s identify the audiences ! Increased use and acceptance of qualitative
research from 1990’s to present
! The complexity of our research problems
today requires understanding trends,
differences, as well as individual stories,
setting
! Individuals advocating for and writing about
mixed methods research as a distinct, new
procedure (e.g., books)

13 14

They may think that it is analyzing


data separately But how do we mix?
Quantitative Data Qualitative Data Converge data:

Qual Results Quan

Connect data:

Qual Quan Results

Mixing: converging the data or connecting


the data
15 16
Why our audience may
recognize it
! The evidence
! Books
! Methodological articles
! Many published research studies using it
! Federal agencies
! Private foundations

17 18

Other writings, initiatives on


mixed methods research: NIH Guidelines
! Research studies reported in journals - Mentioned several
! Methodological articles exploring issues and approaches for combining
procedures qualitative and
quantitative research
! Website for bringing mixed methods writers
together - Considerations for deciding
what model to use (literature
! Conference sessions available, prior studies,
! Handbook of Mixed Methods in Social and realistic design, expertise)
Behavioral Research - Need to describe each
! Private foundation interest; federal agency method thoroughly
interest

19 20
Quotes: National Academy of Sciences
! “Combining qualitative and quantitative Three major research questions in
methods has gained broad appeal in public quality educational research:
health research. The key question has
• What is happening? (qualitative designs)
become not whether it is acceptable or
legitimate to combine methods, but rather • Is there a systematic effect?
how they will be combined to be mutually (a quantitative experiment)

supportive and how findings achieved • Why or how it is happening?


through different methods will be integrated.” (a qualitative followup)
(NIH, Office of Behavioral and Social Science
Research, 1999).

21 22

But even if they recognize it, they may


not appreciate or understand how to We need to define mixed methods
design a mixed methods study research for our audiences
! “We are interested in a randomized control ! Mixed methods research is a design for
trial with a non-experimental approach collecting, analyzing, and mixing both
embedded within it.” (a private foundation quantitative and qualitative data in a
officer) single study or series of studies to
understand a research problem.
! “We accept multi-method studies, but
! The purpose of this form of research is that
investigators mostly do not sort out the both qualitative and quantitative methods, in
complexity of these projects so that we can combination, provide a better understanding
understand them.” (a federal projects of a research problem or issue than either
officer). method alone.
23 24
Now we could mix within single studies or
So how do we design a mixed
multiple studies methods study? The model
Single Study: Worldviews, theoretical frameworks, problem and research
question, skills, resources

Quan Qual Type of mixed methods design

Procedures for:
•designing the title
Results •writing the introduction to a study
•writing the purpose statement and research
Multiple Studies (called multimethod research): questions/hypotheses
Quan Qual Qual Quan •data collection
•data analysis
Study 1 Study 2 Study 3 Study 4 •writing the mixed methods report
•evaluating the mixed methods research
25 26

The next steps in planning our


What is a worldview? study
! Philosophy about your preferences for how ! Let’s write the overall research question
you learn about something through research
for our study
! You prefer the quantitative worldview:
you are the expert, you decide what needs to
be learned, you build in objectivity
! You prefer the qualitative worldview:
participant is the expert, participant helps you
build knowledge, you bring personal bias in
! You prefer both the quantitative and
qualitative worldview
27 28
Then let’s choose a type of mixed Types of mixed methods
methods study to conduct designs
I. Triangulation Mixed Methods Design
! What designs are possible?
QUAN
Data and Results
+ QUAL
Data and Results
Interpretation

II. Nested Mixed Methods Design

QUAN QUAN
Pre-test Post-test
Data and Results Qual Process Data and Results
29 30

Types of mixed methods Triangulation Design:


designs Characteristics

! Collecting both quantitative and qualitative data


III. Explanatory Mixed Methods Design ! Collecting these data at the same time in the
research procedure
QUAN qual ! Analyzing the quantitative and qualitative data
Follow-up Data and
Data and Results separately
Results
! Comparing or combining the results of the
quantitative and qualitative analysis
IV. Exploratory Mixed Methods Design
! Example: collect survey data (quantitative) and
collect individual interviews (qualitative) and then
QUAL quan compare the results
Data and Results Data and Results
Building
31 32
Triangulation Design: When is
it used?
! When you want to combine the advantages
of quantitative (trends, large numbers,
generalization) with qualitative (detail, small
numbers, in-depth)
! When you want to validate your quantitative
findings with qualitative data
! When you want to expand your quantitative
findings with some open-ended qualitative
data (e.g., survey with closed- and open-
ended data)
33

Nested Design: When is it


Nested Design: Characteristics used?
Collecting both quantitative and qualitative data
When you do not have time or
!
!
Collecting both types of data at the same time
resources to commit to extensive
!

! Having ONE form of data play a smaller role in the study


than the other form of data quantitative and qualitative data
! Also, collection
Using one form of data to answer one question; the other form
!
another question ! When you want to study the process of
! Collecting one form of data at one level of analysis and another at an experiment as well as the outcomes
another level of analysis
! Example: You conduct an experiment and during the ! When you want to examine different
experiment you gather qualitative interview data. The levels in an organization
outcomes of the experiment assessed quantitatively
address different questions than the process of the
experiment explored qualitatively. 35 36
Nested Research Design Explanatory Sequential
Design: Characteristics
Experiment
! Viewing the study as a two-phase project
Quan Intervention Quan ! Collecting quantitative data first followed by
Data collection
Data collection
Pre-test Post-test
collecting qualitative data second
! Typically, a greater emphasis is placed on the
Process – collection quantitative data in the study
and analysis of qualitative
data ! Example: You first conduct a survey and then
follow up with a few individuals who
answered positively to the questions through
interviews
37 38

Explanatory Sequential Here is an example of an explanatory design:


Design: When do you use it? Quantitative
Data Collection
(quan)
Quantitative
Data Analysis
(quan)
Case Selection Qualitative
Data Analysis
(QUAL)
Interpretation –
based on quan
and QUAL results

+
! When you want to explain the Qualitative

quantitative results in more depth with


Data Collection

qualitative data (e.g., statistical Quantitative Data* Quantitative Analysis Case Selection Qualitative Analysis Interpretation

differences among groups, individuals Number of cigarettes


CES-D6
Graphic plot of CES
D6 scores over time
- Selected 5 cases
maximally varying
Description of each
case
Why did changes in
smoking occur?

who scored at extreme levels)


for each participant Identified critical Identification of life
Qualitative Data* Graphic plot of months in which events occurring
cigarettes/day values smoking varied during critical
Semi-structured
over time for each months where

When you want to identify appropriate


interviews, audio
participant smoking increased or
recorded and
! transcribed
decreased
Thematic analysis of

participants to study in more depth


life events for each
* Data collected 10 times
case
over the course of a
calendar year for 40 Cross-case thematic

qualitatively participants analysis

Creswell et al. (in progress)


39
Exploratory Sequential Design: Exploratory Sequential Design:
Characteristics When do you use it?
! Viewing the study as a two-phase project ! To develop an instrument when one is
! Qualitative data collection precedes not available (first explore, then
quantitative data collection
develop instrument)
! Typically, greater emphasis is placed on the
qualitative data in the study ! To develop a classification or typology
! Example: You collect qualitative diary for testing
entries, analyze the data for themes, and ! To identify the most important variables
then develop an instrument based on the
themes to measure attitudes on a to study quantitatively when these
quantitative survey administered to a large variable are not known
sample.
41 42

Phase I Qualitative Research - Year 1


Unstructured Interviews -

How will we analyze the quantitative and


Qualitative Data Collection
50 participants
8 observations at the site
16 documents

Qualitative Data Analysis


qualitative data (within the design types)?
Text Analysis: Using QSR N6

Qualitative Findings
Development of codes and themes
for each site Types of analysis:
! Quantitative analysis ! Qualitative analysis
Phase II Quantitative Research - Year 2
Create approximately a 80-item ! Numeric data ! Text/image data
instrument plus demographics
Coding
Quantitative Instrument Development

Administer survey to 500 individuals ! Descriptive trend !

Quantitative Test of the Instrument


Determine factor structure of items and analysis ! Themes
conduct reliability analysis for scales

Quantitative Results
! Hypothesis testing, ! Description
Determine how groups differ
using ANOVA test effect size, interval ! Interrelated themes
estimates

Sequential Exploratory Mixed Methods Design


44
Table. Example of Data Transformation of Text Units into Numeric Data
Count Patients Physicians Medical Assistants
Adj.Count** N=2 N=4 N=4
Row Pct

Triangulation data analysis Column Pct

Themes

Familiarity 13 17 19 49
With the 6.5 4.25 4.75 15.5
QUAN QUAL Form 41.9 27.4 30.7 100.0
5.8 3.0 5.3
data collection data collection
Reactions to 23 100 61 184
the Form 11.5 25.0 15.3 51.75
22.2 48.3 29.5 100.0
10.2 17.4 16.9
• Separate QUAN QUAN QUAL Use for 67 177 36 280
and QUAL data data analysis data analysis Managing 33.5
38.6
44.25
51.0
9.0
10.4
86.75
100.0
analysis Depression
29.8 30.7 10.0
Changes to 115 196 187 498
the Form 57.5 49.0 46.8 153.3
37.5 32.0 30.5 100.0
51.1 34.0 51.7
• Two options
7 86 58 151
• Data transformation (change Results Situational
3.5 21.5 14.5 39.5
Use of the
QUAL to QUAN or QUAN to QUAL) Form 8.9 54.4 36.7 100.0
3.1 14.9 16.1
• Comparison (keep separate and
225 576 361
compare/contrast) 112.5 144.0 90.3
45 100.0 100.0 100.0

Nested data analysis Explanatory sequential data analysis


Quantitative Experiment

Quan Intervention Quan QUAN Qual


data collection Qual
Data collection Data collection data analysis analysis
Pre-test Post-test (purposeful sampling)

• Select cases based on s.d. variables • codes


• Statistical results • themes
Qualitative Process • Outlier cases • Select cases to represent outliers
• Select cases to represent extreme cases • cases
• Extreme cases
• Select cases to make group comparisons
Data
Pre-test Themes/Codes/ Post-test scores
Analysis
scores Interrelated Themes or gain scores

Compare/Describe Results
47 48
Exploratory sequential data analysis

QUAL data analysis Quan data analysis ! Let’s identify how we will analyze the
instrument development data
Quotes
Items on a survey
Codes
Variables on a survey
Themes
Scales on a survey

49 50

Drawing our Design Helpful tips for creating this visual:

! Let’s draw a picture of our design 1. Give a title to the visual model.
2. Choose either horizontal or vertical layout for the model.
! Identify the type of design 3. Draw boxes for quantitative and qualitative stages of data
collection, data analysis and interpretation of the study
results.
! Add in data collection 4. Use capitalized (QUAN) or small letters (quan) to designate
priority of quantitative and qualitative data collection and
! Add in data analysis analysis.
Use single-headed arrows to show the flow of procedures in
Show the flow of activities
5.
! the design.
6. Specify procedures for each quantitative and qualitative data
! Add in “products” for our audiences collection and analysis stage.
7. Specify expected products or outcomes of each quantitative
and qualitative data collection and analysis procedure.
8. Make your model simple.
51 9. Size your model to one page. 52
Now let’s rework our purpose Sample Script for a Concurrent Design
statement using some scripts (Triangulation or Nested)

“The purpose of this concurrent mixed methods study is to


better understand a research problem by converging both
quantitative (numeric) and qualitative (text or image) data.
In this approach, ___________ (quantitative instruments)
will be used to measure the relationship between the
________ (independent variables) and __________
(dependent variables). At the same time in the study, the
__________ (central phenomenon) will be explored using
_____________ (qualitative interviews, documents,
observations, visual materials) with _________ (participants)
at ____________ (the research site).”
53 54

Sample Script for a Sequential Exploratory Design Sample Script for a Sequential Explanatory Design

“The purpose of this two-phase, exploratory mixed methods “The purpose of this two-phase, explanatory mixed methods
study will be to explore participant views with the intent of study will be to obtain statistical, quantitative results from a
using this information to develop and test an instrument with sample and then follow-up with a few individuals to probe or
a sample from a population. The first phase will be a explore those results in more depth. In the first phase,
qualitative exploration of a _______(central phenomenon) by quantitative research questions or hypotheses will address the
collecting ___________(data) from ____________ relationship or comparison of __________ (independent) and
(participants) at _______ (research site). Themes from this ________ (dependent) variables with ___________
qualitative data will then be developed into an instrument (or (participants) at ___________(the research site). In the
survey) so that the __________ (theory and research second phase, qualitative interviews or observations will be
questions/hypotheses) can be tested that ________ (relate, used to problem significant _______(quantitative results) by
compare) ____________ (independent variable) with exploring aspects of the ________ (central phenomenon) with
__________ (dependent variable) for _________(sample of a _______ (a few participants) at ____________ (research
population) at _________ (research site).” 55 site).” 56
Let’s share our drawings of our
Criteria for evaluating our plan: mixed methods procedures
! Use appropriate ! Include a
terminology for title and visual/procedural
design
diagram of methods
! Provide a rationale for
mixing and include it with timeline
early in the study ! Use rigorous procedures
(“when you use…”) for the quantitative data
! Create a mixed methods collection and analysis
purpose statement
! Identify types of qual
and quan data to be
collected and qual and
quan data analysis
steps
57 58

How to Design a Mixed


Methods Study

by John W. Creswell, Ph.D. and Vicki L. Plano Clark, M.S.


University of Nebraska-Lincoln

Andrews University, July, 2004