Mixed Methods Research

Introduction

2
 A Definition of Mixed Methods
Research

.. as a method, it focuses on collecting, analysing,

& mixing both quantitative & qualitative data in a
single study or series of studies. Its central
premise is that the use of quantitative &
qualitative approaches, in combination, provides a
better understanding of research problems than
either approach alone.
 How would you combine two types of data?

QUALITATIVE QUANTITATIVE
Text Data Numeric Data

This is a sample of a text 2 3 4 2 5 2 3 1 1 2 3 4 2 3 3

file of words that might 2 1 1 1 5 3 4 1 2 3 1 4 4 5 5
be collected on interview 4 1 2 1 4 3 3 5 1 4 2 3 1 5 5
transcripts, observation 2 2 1 5 3 5 1 3 1 5 3 2 2 5 1
field notes, or optically- 3 2 4 4 3 1 2 4 2 2 4 1 5 5 4
scanned documents.
2 1 5
Framework for viewing perspectives on
mixed methods

Quantitative Data Qualitative Data

Mixed Methods

Method
Methodology
Paradigm
Perspective Use of mixed methods
in other designs 5
Strength & weakness of quantitative &
qualitative methods

Quantitative Qualitative

Generalization

Large sample Small sample

Details, in depth

6
Six core characteristics of mixed
methods research

1. The collection of both qualitative and quantitative data

(open- and closed-ended) in response to research
questions.
2. The analysis of both qualitative & quantitative data.
3. Persuasive and rigorous procedures for the qualitative
and quantitative methods.
4. The integration of these two data sources (merging,
connecting, embedding).
5. The use of a specific mixed methods design that
involves a concurrent or sequential integration (equal or
unequal emphases).
6. An approach to research that has a philosophical
foundation
7
A mixed methods researcher
 Collecting both
quantitative & qualitative data

Quantitative data:
Qualitative data:

i. Instruments
i. Interviews
ii. Checklists
ii. Observations
iii. Records
iii. Documents
iv. Audio-visual materials

9
 Data analysis

Quantitative: Qualitative:

i. Use statistical i.Use text and images,

analysis,
ii.For coding
ii. For description
iii.For theme
iii. For comparing groups development
iv. For relating variables iv.For relating themes

10
Mixing or linking the data

Converge data:

Quali Results Quanti

Connect data:

Quali Quanti Results

Embed the data:

Quanti data
Quali data 11
Typical situations in which mixed
methods is used to
What is the reason for using mixed
methods?

i. The insufficient argument either quantitative or

qualitative may be insufficient by itself
ii. Multiple angles argument quantitative and qualitative
approaches provide different pictures
iii. The more-evidence-the-better argument combined
quantitative and qualitative provides more evidence
iv. Community of practice argument mixed methods may
be the preferred approach within a scholarly community
v. Eager-to-learn argument it is the latest methodology
vi. Its intuitive argument it mirrors real life
 Steps in the process of designing a
mixed methods study

Designing a mixed methods study:

1. Preliminary considerations
2. Creating a title
3. Posing a general question
4. Listing the types of data collection & analysis
5. Making explicit your worldview
6. Identifying your research design
7. Drawing a figure of your design
8. Writing a purpose statement
9. Writing research questions
10. Completing a research plan
 Preliminary considerations
(before you begin to design)

1. Research problem
i. Content any topics
ii. Fit the problem to mixed methods (arguments)

2. Access to both qualitative and quantitative data

3. Background and resources

4. Receptive audience

15
 Pose the general question to be
answered

1. Write it as a question
2. Look to see how it is phrased
3. Make sure that it is specific enough and focused
(an answerable question)
4. Ask yourself, when I end the study, what question
would like to have answered?

16
 List your types of data collection
(a review)

17
List your approach to data
analysis (a review)

Quantitative analysis: Qualitative analysis:

Use statistical Use text and

analysis, images,
For description For coding
For comparing For theme
groups development
For relating For relating
variables themes
Design-type Design-type

18
Determine your worldview (philosophical
developments)

Postpositivism Constructivism
Determination Understanding
Reductionism Multiple participant
Empirical observation and meanings
measurement Social and historical
Theory verification construction
Theory generation
Advocacy/Participatory Pragmatism
Political Consequences of actions
Empowerment issue- Problem-centered
oriented Pluralistic
Collaborative Real-world practice
Change-oriented oriented
19
 several stances on philosophy in
mixed methods

1. One paradigm (pragmatism, transformative)-(Tashakkori

& Teddlie, 2003; Mertens, 2003)
2. Multiple paradigms (dialectic perspective)-(Greene, 2007)
3. Linking paradigms to design features)-(Creswell & Plano
Clark, 2007)
4. Epistemological stance (ontology, epistemology, axiology,
methodology)-(Guba & Lincoln, 2005)
5. Shared beliefs in a research field-(Morgan, 2007)

What it is?
How it informs your study?
20
Make explicit your interpretive lens
(theory)

1. Advocacy lens (feminist, racial, ethnic, disability,

sexual orientation) (Mertens, 2003)
2. Social science lens (social science theory)

3. Components:
a. What is it
b. Who has used it in your field
c. How it will shape your study (rephrase your guiding
research question, if a lens applies)
21
 State your procedures (methods)

1. Procedures for handling your qualitative &

quantitative data
2. Sequence concurrent or sequential or both
3. Emphasis emphasis on qualitative or quantitative
4. Sometimes both concurrent & sequential phases
are used
5. Designs may include more than two phases
6. Think about using a simple, elegant design

22
Parsimonious designs (Creswell & Plano Clark, 2007)

Concurrent Mixed Methods Designs

Triangulation Design

QUAN
QUAN QUAL
QUAL
Data
Data&& Data
Data&&
Interpretation
Results
Results
Interpretation Results
Results

Embedded Design

QUAN Intervention QUAN

QUAN QUAN
Pre-test
Pre-test Post-test
Post-test Interpretation
Interpretation
Data
Data&& Data
Data&&
qual
qual
Results
Results Results
Results
Process
Process
23
 Sequential
designs
Explanatory Design
QUAN
QUAN qual
qual
Data
Data&& Data
Data&&
Interpretation
Results
Results Following up
Results
Results Interpretation

Exploratory Design
QUAL
QUAL quan
quan
Data
Data&& Data
Data&& Interpretation
Results
Results Building to
Results
Results Interpretation

Sequential Embedded Design

Before-
QUAN
QUAN After-
Before- Intervention After-
intervention Intervention
intervention Trial intervention
Trial intervention
qual
qual qual Interpretation
Interpretation
qual
24
Design options
Design Name Equal priority QUAN emphasis QUAL emphasis

Concurrent, QUAL+QUAN QUAN+qual QUAL+quan

triangulation
Concurrent, n/a QUAN(qual) QUAL(quan)

embedded
Explanatory, QUANQUAL QUANqual quanQUAL

sequential, quan first

Exploratory QUALQUAN qualQUAN QUALquan

sequential, qual first

Sequential, n/a (qual) QUAN (quan) QUAL

embedded QUAN (qual) QUAL (quan)

25
Explanatory sequential design
Quantitative Case Selection Qualitative Interpretation
Qua itative
Data Collection Data Analysis Data Analysis
nt
(quan) (QUAL) based on quan
& QUAL results
+
Qualitative
Data Collection

Quantitative Data* Quantitative Analysis

Case Selection Qualitative Analysis Interpretation
Number of cigarettes Graphic plot of-CES Selected 5 cases Description of each Why did changes in
CES-D6 D6 scores over time maximally varying case smoking occur?
for each participant Identified critical Identification of life
Qualitative Data* Graphic plot of months in which events occurring
Semi -structured cigarettes/day valuessmoking varied during critical
over time for each months where
interviews, audio
participant smoking increased or
recorded and
decreased
transcribed
Thematic analysis of
life events for each
* Data collected 10 times
case
over the course of a
calendar year for 40 Cross-case thematic
participants analysis

Source: Creswell, Plano Clark, Shope, McVea. (in progress)

 Phase I Qualitative Research - Year 1

Qualitative Data Collection Unstructured Interviews -

50 participants
8 observations at the site
16 documents

Qualitative Data Analysis Text Analysis: Using QSR N6

Development of codes and themes

Qualitative Findings for each site

Phase II Quantitative Research - Year 2

Create approximately a 80-item
Quantitative Instrument Development instrument plus demographics

Administer survey to 500 individuals

Quantitative Test of the Instrument
Determine factor structure of items and
conduct reliability analysis for scales

Quantitative Results
Determine how groups differ
using ANOVA test

Exploratory sequential design

Embedded research design

Experiment
Intervention
QUANTI QUANTI
Data collection Data collection
Pre-test Post-test

Process
collection
& analysis of qualitative
data 29

(before, during, after trial)

Mixing the quanti & quali data
Type of Type of Why Mixing Where Mixing Occurs
Mixing Design Occurs in Research Process

Connecting Sequential One phase Between data analysis

builds on the (Phase 1) and data
other collection (Phase 2)

Merging Concurrent Bring results After analysis of both

together quan and qual
typically in discussion

Embedding Sequential or Either Either between phases

Concurrent building or or in discussion after
bringing analysis
results
together 30
Methodological issues
Concurrent Designs
Use strategies to explore
contradictory findings
Use parallel questions
Select sub-sample of
quantitative for qualitative
Be sensitive to bias from one
data collection to the other
Sequential designs
In Explanatory Design, select
qual sub-sample from quan
sample
In Explanatory Design,
consider alternatives for
followup qual sampling
In Exploratory Design,
samples can differ
In Exploratory Instrument
Design, consider qual data
analysis approaches for
developing instrument
31
Write a purpose statement for a
triangulation design

This mixed methods study will address _________________ (overall content-

aim of the study). A triangulation mixed methods design will be used, and
it is a type of design in which different but complementary data will be
collected on the same topic. In this study, _________________ (quantitative
instruments) will be used to test the theory of _____________ (the theory)
that predicts that __________________ (independent variables) will influence
________________ (positively, negatively) the __________ (dependent variables
or outcomes) for ____________ (participants) at __________ (the research
site). Concurrent with this data collection, qualitative _______________
(type of qualitative data, such as interviews) will explore ____________ (the
central phenomenon) for _________________ (participants) at
_____________(site). The reason for collecting both quantitative and
qualitative data are to bring together the strengths of both forms of
research to ___________________ (e.g., compare results, validate results,
corroborate results).

32
Write a purpose statement for a
embedded design
This mixed methods study will address _________________ (overall
content aim of the study). An embedded mixed method design
will be used, and it is a design in which one data set provides a
supportive, secondary role in a study based primarily on the
other data set. The primary purpose of this study will use
________________ (quantitative instruments) to test the theory of
_____________ (the theory) that predicts that __________________
(independent variables) will influence ________________ (positively,
negatively) the __________ (dependent variables or outcomes) for
____________ (participants) at __________ (the research site). A
secondary purpose will be to gather qualitative data
_______________ (type of qualitative data, such as interviews) that
will explore ____________ (the central phenomenon) for
_________________ (participants) at _____________(site). The
reason for collecting the secondary database is ________________
(e.g., to address different question, to provide support for the
primary purpose).

33
 Write a purpose statement for an
explanatory design
This study will address _______________ (content-aim of the study).
An explanatory mixed methods design will be used, and it will
involve collecting qualitative data after a quantitative phase in order
to explain or follow up on the quantitative data in more depth. In
the first quantitative phase of the study, ______________ instrument
data to be collected from _______________ (participants) at
___________ (research site) to test _______ (the theory) that explains
why ______________ independent variables) relate to the ____________
(dependent variables). The second qualitative phase will be
conducted because ______________ (intent of the qualitative phase).
In this exploratory follow-up, the ______________ (central
phenomenon) will be tentatively explored with ___________
(participants) at _____________ (the research site). The reason for the
exploratory follow-up is to _______________ (e.g., to help explain or
build upon initial quantitative results).

34
Write a purpose statement for an
exploratory design
This study addresses _________________ (content-area of the study).
The purpose of this exploratory sequential design will be to
__________________ (e.g., develop an test an instrument, generate a
taxonomy). The first phase of the study will be a qualitative
exploration of ________________ (the central phenomenon) by
collecting ___________________ (types of data) from _________________
(participants) at _____________ (the research site). The second
quantitative phase will follow up on the qualitative phase for the
purpose of __________(intent of this followup). In the quantitative
phase, ___________ (instrument data) will be collected from
____________ (participants) at ______________ (research site).
Quantitative research questions/hypotheses will be formulated after
the completion of the initial qualitative phase. The reason for
collecting qualitative data initially is that _____________________ (e.g.,
instruments are not available, variables are not known, there is little
guiding theory).
35
 Qualitative research questions

Qualitative central question:

1. Begin with what or how

2. Focus on single phenomenon
3. Use exploratory verbs (discover, understand, explore)
4. Non-directional language
5. A general question (allowing participants perspectives
to emerge)

36
 Quantitative research questions

Quantitative central question:

1. Can be hypotheses or questions

2. State variables independent, dependent, mediating,
covariates
3. Develop from theory
4. Use distinct measures for independent and
dependent variables
5. Order variables from independent to dependent

37
Writing research questions/hypotheses in
mixed methods research

1. Write qualitative research questions & write

quantitative research questions/hypotheses

2. Also write a mixed methods research question

3. Write these questions separately

38
 A new type of research question: A mixed
methods question

Three ways to write this question:

Methodologically-focused:

To what extent do the qualitative results confirm the quantitative

results?

Content-focused:

How do the interviews with adolescent boys support the quantitative

results that their self-esteem changes during the middle school years?

Hybrid of quantitative and qualitative elements:

What results emerge from comparing the exploratory qualitative data

about boys self-esteem with outcome quantitative instrument data
measured on a self-esteem instrument?
Order the topics for your plan

1. Title
2. Introduction
i. (Problem)
ii. Worldview/theoretical lens
iii. Audience
iv. Purpose Statement
v. Research Questions
3. Literature Review
4. Methods
i. Type of Mixed Methods Design (also add definition of mixed methods)
ii. Types of Data Collection
iii. Types of Data Analysis
iv. Sequence/Emphasis/ Mixing Procedures
v. Figure of procedures
vi. Anticipated methodological issues
5. Ethical issues anticipated
6. Validity issues
7. Researcher resources and skills 40
8. References, Appendices
Benefits of Mixed Method

1 Triangulation:
Seeks convergence, corroboration and correspondence of results from
different methods.
2 Complementarity:
Seeks elaboration, enhancement, illustration, clarification of results of one
method with the results from the other method.
3 Development:
Seeks to use the results of one method to help or inform the other method.
4 Initiation:
Seeks the discovery of paradox and contradiction, new perspectives, the
recasting of questions or the results from one method with questions or
results from the other method .
5 Expansion:
Seeks to extend the breadth and range of inquiry by using different
methods from different inquiry components.
41
 Reference

Creswell, J. W. & Plano Clark, V. L. (2011). Designing and conducting

mixed methods research (2nd ed.). Thousand Oaks, CA: Sage
Publications, Inc.

Creswell, J. W. (2010). Mapping the Landscape of Mixed Methods

Research. Department of Educational Psychology, University of
Nebraska-Lincoln.

42