Lecture # 2

The Research Process

Research: A way of Thinking
The Research Process: A Quick Glance
1. Introduction

1.What is Research ??
i. One of the ways to find answers to your

Research Methods: Dr. Imran Usman

questions
2.It is implication of a process, and the process:
i. Is within a framework of a set of philosophies
ii. Uses procedures, methods, and techniques that
are tested for their validity and reliability
iii.Is designed to be unbiased and objective


2.i. Possible set of Philosophies
 Your philosophical orientation may stem from
one of the several philosophical approaches
in research:

Research Methods: Dr. Imran Usman

i. Positivist
ii. Interpretive
iii. Phenomenolist
iv. Action oriented
v. Participatory
vi. Qualitative
vii. Quantitative
2.ii Validity and Reliability
 Deals with whether the procedures, methods,
and techniques adopted are tested for their
validity and reliability

Research Methods: Dr. Imran Usman

1.Validity:
 Ensures that in a research study correct
procedures have been applied to find
answers to a question
2.Reliability:
 Refers to the quality of a measurement
procedure that provides repeatability and
accuracy
2.iii Unbiased and Objective
1.Unbiased:
 Thatyou have taken each step in an unbiased

Research Methods: Dr. Imran Usman

manner and drawn each conclusion to the
best of your ability and without introducing
your own vested interests.
2.Objective:
 Should achieve a certain objective for which the
study is focused on.
3. An Overview of Research Methods
and Methodologies

Why Do I Need to Know About Different



Methods?

Research Methods: Dr. Imran Usman



 As a graduate student...
 To be able to read and understand the empirical
literature in your field; to become a critical consumer
of information.

 As a graduate student preparing for a thesis or
dissertation…
 To be able to both design and implement your thesis
or dissertation as well as future studies that interest
you.

 As a future practitioner…
 To be able to intelligently participate in research

Research Methods: Dr. Imran Usman

projects, evaluations, and studies undertaken by
your institution.

 As an educated citizen ...
 To understand the difference between scientifically
acquired knowledge and other kinds of information.

What’s the Difference Between
“Method” and “Methodology”?

Method:
 Methodology:

 Techniques for  The underlying theory

gathering evidence and analysis of how
 The various ways of research does or
proceeding in should proceed,
gathering often influenced by
information discipline



Epistemology, Methodology, and
Method
 a research method is a technique for (or way
of proceeding in) gathering evidence“

Research Methods: Dr. Imran Usman

 while "methodology is a theory and analysis of
how research does or should proceed”
 and "an epistemology is a theory of
knowledge”


Research Methods: Dr. Imran Usman



"It is the theory that decides what can be



observed."


 Albert Einstein

An Overview of Empirical Research
Methods

Descriptive

Experimental


(Qualitative) (Quantitative)
 Ethnography
 True Experiment
 Case Study
 Quasi-Experiment
 Survey/Sampling
 Focus Groups  Meta-Analysis

 Discourse/Text Analysis 
 Quantitative
Description
 Prediction/Classification


Ethnographies
+Observational field work done in the actual
context being studied
+Focus on how individuals interrelate in their
own environment (and the influence of this
environment)
- Difficult to interpret/analyze
- Time consuming/expensive
- Can influence subject behavior
Case Studies
+Focus is on individual or small group
+Able to conduct a comprehensive analysis
from a comparison of cases
+Allows for identification of variables or
phenomenon to be studied
- Time consuming
- Depth rather than breadth
- Not necessarily representative
Survey Research
+An efficient means of gathering large amounts
of data
+Can be anonymous and inexpensive
- Feedback often incomplete
- Wording of instrument can bias feedback
- Details often left out
Focus Groups
+Aid in understanding audience, group, users
+Small group interaction more than individual
response
+Helps identify and fill gaps in current
knowledge re: perceptions, attitudes,
feelings, etc.
- Does not give statistics
- Marketing tools seen as “suspect”
- Analysis subjective
Discourse/Text Analysis
+Examines actual discourse produced for a
particular purpose (job, school)
+Helps in understanding of context, production,
audience, and text
+Schedule for analysis not demanding
- Labor intensive
- Categories often fluid, making analysis difficult
Quantitative Descriptive Studies
+Isolates systematically the most important
variables (often from case studies) and to
quantify and interrelate them (often via
survey or questionnaire)
+Possible to collect large amounts of data
+Not as disruptive
+Biases not as likely
- Data restricted to information available
Prediction and Classification Studies
Goal is to predict behaviors:


 Prediction forecasts and interval variable

(Diagnostic/TAAS scores)
 Classification forecasts a nominal variable (Major
selection after taking 2311)
+ Important in industry, education to predict
behaviors
- Need substantial population
- Restricted range of variables can cause
misinterpretation
- Variables cannot be added together; must be
weighted and looked at in context of other
variables
Positive Aspects of
Descriptive/Qualitative Research
 Naturalistic; allows for subjects to interact
with environment
 Can use statistical analysis

 Seeks to further develop theory (not to

influence action); Prescientific
 Coding schemes often arise from interplay
between data and researcher’s knowledge of
theory

Problems with Descriptive/Qualitative
Research
 Impossible to overlay structure
 Impossible to impose control

 Subject pool often limited, not representative

 Seen as more “subjective,” less rigorous

 Beneficial only in terms of initial investigation

to form hypothesis
Experimental Research: True
Experiment
+Random sampling, or selection, of subjects
(which are also stratified)
+Introduction of a treatment
+Use of a control group for comparing
subjects who don’t receive treatment
with those who do
- Adherence to scientific method (seen as
positive, too)
- Must have both internal and external
validity
- Treatment and control might seem artificial
Experimental Research: Quasi-
Experiment
+Similar to Experiment, except that the
subjects are not randomized. Intact
groups are often used (for example,
students in a classroom).
+To draw more fully on the power of the
experimental method, a pretest may be
employed.
+Employ treatment, control, and scientific
method
- Act of control and treatment makes
situation artificial
- Small subject pools
Meta-Analysis
+Takes the results of true and quasi-
experiments and identifies
interrelationships of conclusions
+Systematic
+Replicable
+Summarizes overall results
- C/C apples and oranges?
- Quality of studies used?
Positive Aspects of Experimental
Research
 Tests the validity of generalizations
 Seen as rigorous

 Identifies a cause-and-effect relationship

 Seen as more objective, less subjective

 Can be predictive
Problems with Experimental Research
 Generalizations need to be qualified according
to limitation of research methods employed
 Controlled settings don’t mirror actual
conditions; unnatural
 Difficult to isolate a single variable

 Doesn’t allow for self-reflection (built-in)

Assessing Methods
 Research Question(s) is/are key
 Methods must answer the research question(s)

 Methodology guides application

 Epistemology guides analysis

 All must include “rigor”

Testing the Waters
 How do you come up with a good research
question?
 How do you determine if the method you plan
to use will answer your question?
 What epistemology should you use to analyze
data?
Case Scenario
 Test your research savvy with the following
case. Assume that you are the Mayor of
Greenwood, a small town in Illinois, and
you’ve got to make a decision based on the
information collected from the following
research study.
Crime Reduction Program, City of
Greenwood
 The chief of police wants to
experiment with increasing the
number of patrol officers (X) to
reduce the crime rate (Y).
 The chief invites all twelve-
precinct captains to participate
in the experiment; only the
103rd volunteers.
 In October, patrol officers in the
103rd are increased by 15%.
103rd  Reported crime drops 5%
between September &
December. The chief now
wants to implement the
program citywide.
You are the mayor. Would you
support this request based upon the
results of this study?

ü Could severe weather in November and December

have caused the crime rate to decline?
ü Is crime seasonal, peaking in the summer and
declining in the winter?

More Problems

ü Since the captain of the 103rd volunteered for the

program, could he have already implemented other
programs that account for the decline in crime?
ü Since the officers in the 103rd knew they were involved
in a priority program, is it possible that they recorded
reported crime differently?
ü
More Problems
ü Will the crime reduction impact last very long?
ü Could random error in the measurement of the crime
rate account for the difference?
ü Was the crime rate in the entire city going down
anyway?
ü

What Makes Research Good?
 Validity
 Reliability

 Replicability

 Consistent application/analysis

 “Trustworthiness”

 Rigor
Validity in Research
 Refers to whether the research actually
measures what it says it’ll measure. Validity is
the strength of our conclusions, inferences or
propositions.

 Internal Validity: the difference in the dependent
variable is actually a result of the independent
variable

 External Validity: the results of the study are
generalizable to other groups and environments
outside the experimental setting

 Conclusion Validity: we can identify a relationship
between treatment and observed outcome

 Construct Validity: we can generalize our
conceptualized treatment and outcomes to
Reliability in Research
 The consistency of a measurement, or the
degree to which an instrument measures the
same way each time it is used under the same
condition with the same subjects. In short, it is
the repeatability of your measurement. A
measure is considered reliable if a person's
score on the same test given twice is similar. It
is important to remember that reliability is not
measured, it is estimated. Measured by
test/retest and internal consistency.
Validity and Reliability
 The relationship between reliability and
validity is a fairly simple one to understand: a
measurement can be reliable, but not valid.
However, a measurement must first be
reliable before it can be valid. Thus reliability
is a necessary, but not sufficient, condition of
validity. In other words, a measurement may
consistently assess a phenomena (or
outcome), but unless that measurement tests
what you want it to, it is not valid.

Rigor in Research
 Validity and Reliability in conducting research
 Adequate presentation of findings:
consistency, trustworthiness
 Appropriate representation of study for a
particular field: disciplinary rigor
 Rhetorical Rigor: how you represent your
research for a particular audience
Key Considerations to Design Your
Research Approach
 What question do you want to answer?
 For what purposes is the research being done?
i.e., what do you want to be able to do or
decide as a result of the research?
 Who are the audiences for the information
from the research, e.g., teachers, students,
other researchers, members of a disciplinary
community, corporate entities, etc.?
 From what sources should the information be
collected, e.g., students, teachers, targeted
groups, certain documentation, etc.?
Key Considerations to Design Your
Research Approach
 What kinds of information are needed to make
the decisions you need to make and/or to
enlighten your intended audiences, e.g., do
you need information to really understand a
process, the students who engage in a
process, strengths and weaknesses of a
curriculum or program, benefits to students
or institution or agency, how aspect of a
program are problematic, etc.?



Key Considerations to Design Your
Research Approach
 How can that information be collected in a
reasonable fashion, e.g., questionnaires,
interviews, examining documentation,
observing staff and/or clients in the
program, conducting focus groups among
staff and/or students, etc?
 How accurate will this information be?
 When is the information needed (so, by when
must it be collected)?
 What resources are available to collect the
information?
 How will this information be analyzed?

The Importance of Methods and
Methodology
 “The most common error made in reading
[and conducting] research is overlooking the
methodology, and concentrating on the
conclusions. Yet if the methodology isn’t
sound, the conclusions and subsequent
recommendations won’t be sound.”