PARTICIPANTS AND DATA

COLLECTION: IDENTIFYING HOW

QUANTITATIVE INFORMATION IS
GATHERED

1
 How Do You Identify the Participants and Data
Collection in a Quantitative Study?
• Researchers describe the procedures that they use (i.e., their
method) to conduct their quantitative studies in the Method
section of research reports.
• The Method section often starts with a brief description of the
study’s research design.
• However, most of the information you read in the Method
section focuses on how the researcher actually gathered
information from people or organizations.
• Because the collection of data is central to conducting
research, researchers generally include the same categories of
information when discussing their data collection methods

2
 How Do You Identify the Participants and Data
Collection in a Quantitative Study?
• Let’s consider these categories by first reading an excerpt
from a quantitative research report by Frankenberger (2004)
• As you read this passage, think about what the author conveys
about the data collection.
• Method
• Participants
• Participants were 215 male (52%) and female (48%) high school students ranging
in age from 14 to 18 ( M =15.91, SD =1.14). Because prior research on adolescent
egocentrism has historically used White, middle-class samples, data were collected
at a high school in the predominantly middle-class neighborhood of a medium-sized
city in the Pacific Northwest. Students were predominantly White (77%). Of the
original 223 surveys administered, 8 were excluded from the analysis because of
missing data.

3
• Procedure
• Students completed a self-administered questionnaire during one of their health classes. The
questionnaire was titled “You and Your Opinions About Cigarette Smoking” and included
measures of adolescent egocentrism, sensation seeking, risk perceptions, and questions
regarding smoking behavior. Participants were informed that the results of the survey were
anonymous and confidential. Participation was voluntary.

• Instruments
• Adolescent egocentrism . Adolescent egocentrism was measured with the AES [Adolescent
Egocentrism Scale] developed by Enright and colleagues (Enright et al., 1979; Enright et al.,
1980). The scale consists of three five-item subscales for the imaginary audience, personal
fable, and self-focus. For each item, participants are asked to rate the importance of a
statement on a scale ranging from 1 = not important to 5 = very important. The AES has
demonstrated acceptable to good levels of internal consistency ranging from Cronbach’s alpha
5 .78 (Enright et al., 1979) to Cronbach’s alpha 5 .83 (Enright et al., 1980). Scores were
calculated by summing across the five items for each subscale. . . . ( Frankenberger, 2004 , pp.
580 – 581 )

4
 Explanation of Excerpts
• We learn that the author collected data from 215 students at one high
school in the Pacific Northwest. We also learn why she selected this school
because the students had characteristics similar to those used in past
research on this topic. [Sites and participants]
• The author next described her general procedures for data collection. The
procedures included having students complete a questionnaire during health
class. In addition, we find that the author was concerned with how she
interacted with her participants and made sure she treated them ethically.
[General procedure for collecting data]
• Frankenberger provided specific details about the instruments used to
collect data (Adolescent Egocentrism Scale), who developed it (Enright
and colleagues), the types of responses collected with this instrument
(rating of importance from 1 to 5), and information about the scores from
this instrument (e.g., acceptable to good levels of internal consistency).
5
 How Do You Understand the Selection of Sites
and Participants in a Quantitative Study?
• One of the most important elements to consider when reading
any research study is the information provided about the
study’s participants. The participants are the entities who take
part in the research.
• Individuals, Couples, families, organizations, agencies, states,
and countries are all examples of “participants” in research.
• Information about the participants in a quantitative study
typically appears as one of the first topics discussed in the
Method section, often under a subheading such as Participants
or Sample

6
 How Do You Understand the Selection of Sites
and Participants in a Quantitative Study
• To understand and evaluate the participants in a quantitative
study, you should identify three kinds of information as you
read: the study’s population and sample, the sampling strategy
used, and the size of the sample
• Identify the Population and Sample
• The general group of interest in a quantitative study is referred to as the
population. A population is a group of individuals or organizations who
have the same characteristic. Populations can be small, such as all
administrators in one school district. Usually the population for a
quantitative research study is large, such as all voters in a community, all
girls participating in high school sports in a city, or all child services
agencies in a state.

7
 How Do You Understand the Selection of Sites
and Participants in a Quantitative Study
• Although quantitative researchers conduct their studies to learn
about a population, they typically do not collect data from
every person or organization in the group. In most situations, it
simply would take too long, cost too much, or be too difficult
to locate everyone in a group. Therefore, researchers select
individuals who are part of the population to participate in
their quantitative studies.
• A sample is a subgroup of a population that participates in a
study and provides data for the study. As illustrated in Figure 1
, researchers can select different possible samples from a
population of interest. For example, a researcher interested in
elementary principals (the population) may select a subset of
principals from around the state as the sample for a research 8
study
FIGURE 1 A Population and Examples of Possible Samples

9
• Determine the Sampling Strategy That is Used
• Researchers employ different approaches for selecting the
samples in their studies, which are called sampling strategies.
• The two categories of sampling strategies are probability and
nonprobability sampling.
• Table 1 , lists several types of probability sampling. The most
common type of probability sampling is a strategy called
random sampling , where the researcher selects participants at
random from all the members of the larger group.

10
TABLE 1 Types of Sampling Strategies Found in Quantitative Research

11
 Probability sampling.
• Probability sampling occurs when researchers use a
random process to select individuals (or units, such as
schools) from the population so that each individual
has a known chance (or probability) of being
selected.
• The use of probability sampling in a study has the
advantage that any bias that may exist in the
population should have been equally distributed
among the selected participants.

12
 Probability sampling
• This is the most rigorous form of sampling in quantitative
research because the investigator can make the strongest
claims that the sample is representative of the population.
• The use of a probability sampling strategy is particularly
important in studies using a survey design because the purpose
is to describe trends in a population.
• Despite its advantages, it is not always possible for researchers
to use probability sampling in their studies.
• Probability sampling is also unusual in experimental studies.

13
 Nonprobability sampling
• Nonprobability sampling occurs when researchers
select individuals to study because they are available,
convenient, and meet some criteria or characteristics
that the investigator seeks to study.
• The two most common types of nonprobability
sampling are convenience and purposive sampling, as
summarized in Table 1 .

14
 Exercise
• Read the following passage from this study. What type of
• sampling strategy did the authors use? Is the sample
representative?
– Students in five middle schools located throughout the State of New
Jersey in the United States provided the data for this research. Each
school contained Grades 6 through 8 and almost all students were
between the ages of 11 and 14. The five schools were from an initial
group of seven middle schools in the state that had chosen to participate
in an online survey of their students regarding bullying in the late
spring of 2006.

15
 Solution To Exercise
•From this passage, we can conclude that the authors used a
nonprobability sampling strategy to select students who were
convenient at a few middle schools. These middle schools
were convenient because they were willing to give the
researchers access to their students to study bullying issues
and were all located in one region. Therefore, the schools and
sampled students are likely not representative of all middle
school students. This means that the results may not generalize
beyond the students that were studied.

16
Determine Whether the Sample Included a Large
Number of Participants
• The final essential detail that you should look for when
reading about the participants in a quantitative study is the size
of the study’s sample. The sample size is the number of
participants (or organizations) that actually participated in the
study and provided data. Researchers report their sample sizes
by stating the number in the text or using a shorthand such as “
N =52,” where N stands for the “number of participants”
• A general rule of thumb for quantitative research is that the
larger the size of the sample, the better. the larger the sample,
the less likely there is a sampling error in the results.

17
 Determine Whether the Sample Included a Large
Number of Participants
• The second reason that it is necessary for quantitative
studies to have included a large sample size is that it
makes it more likely that the researcher, after
analyzing the gathered data, can find that variables
are related or a treatment made an impact.
• The best way for researchers to pick their sample size
is to use precise estimates of the number of
participants needed as derived from sample size
tables or special formulas that have been developed.

18
 Determine Whether the Sample Included a Large
Number of Participants
• You can feel more confident that the sample is large enough
when a researcher reports using procedures like a sample size
formula to determine the size of their sample.
• The size of the sample that is appropriate for a particular study
depends on many considerations, including the overall size of
the population, the number of variables of interest, and the
type of quantitative research question that needs to be
answered.

19
Some general guidelines for the minimum sample
sizes required for basic versions of different designs
• a minimum of 15 participants in each group in a true
experiment or quasi- experiment,
• ■ one participant (or a few) in a single-subject study (with data
collected over a large number of points in time),
• ■ a minimum of 30 participants for a correlational study that
relates variables, and
• ■ approximately 350 individuals (or more) for a survey study,
but this size varies depending on several factors, including the
size of the overall population. (Clark and Creswell, 2014)

20
 Exercise
• Consider the following passage from a quantitative survey
study. What is the sample size? What sampling strategy
appeared to be used to select the participants? What population
does the sample represent?
– A random sample of 350 was drawn from a list of 8,506 active elementary
public school principals enrolled as members of the National Association of
Elementary School Principals. . . . Surveys were returned from principals
representing various regions of the United States: Midwest (32%), Northeast
(27%), South (26%), and West (15%). These proportions closely reflected the
membership of the population from which the sample was drawn. . . .
Respondents ( N = 214) reported the length of their service as elementary
school administrators to be as follows: under 1 to under 5 years (28.5%), 5 to
under 10 years (30.8%), and 10 or more years (40.7%). ( Abril & Gault, 2006 ,
p. 11 )

21
 Solution to exercise
• From this passage, we learn that Abril and Gault (2006)
contacted 350 individuals, but the actual sample size of
individuals who participated was 214. That is, N = 214. The
authors used probability sampling to identify their participants
by randomly selecting them from a list of all principals who
belong to a national organization. Therefore, the population of
this study is all principals who belong to National Association
of Elementary School Principals in the United States. Because
of the extensive membership of this organization and its
representation of all areas of the United States, this sample is
likely representative of all U.S. elementary principals.

22
 How Do You Understand the Instruments Used to
Gather Data in a Quantitative Study?
• When reading about a quantitative research study, you should
expect that the researchers specified their variables,
determined the type of instrument needed to collect the
information for each variable, and selected instruments that are
good quality before gathering any data.
• The process researchers use to select their instruments is
summarized in Figure 2 using a variable called “test anxiety.”

23
FIGURE 2 The Flow of Activities That Researchers Use to
Collect Quantitative Data

24
Identify How the Researcher Specified the Variables
• To understand what data are collected in a study, you
first need to identify the variables that are of interest
from the study’s research questions and hypotheses.
• These variables include independent, dependent, and
control variables.
• Variables represent characteristics or attributes that
vary among participants

25
 Identify the Type of Instrument Used to Gather
Information
• Researchers use various types of instruments to collect
quantitative data for their variables.
• An instrument is a tool used to gather quantitative data by
measuring, observing, or documenting responses to specific
items.
• There are five general types of instruments that you find when
reading quantitative studies. A summary of each type with
examples of items that might be included in such an
instrument is illustrated in Figure 7.3 .

26
FIGURE 3 Types of Instruments, Their Definitions, and
Examples of Items

27
Assess the Evidence That the Researcher Used
a Good Instrument
• When you read about the instruments used in a
quantitative study, first look for information
indicating that the scores from each instrument are
reliable.
• Reliable means that scores from an instrument are
stable and consistent. The scores should be nearly the
same when researchers administer the instrument
multiple times to the same participants.

28
 Assess the Evidence That the Researcher Used a
Good Instrument
• Another aspect of reliability is that scores from the
multiple items on an instrument need to be consistent
with each other.
• When an individual answers certain questions one
way, the individual should consistently answer
closely related questions in a similar way.
• You can identify evidence of reliable scores by
recognizing the following kinds of information as you
read:

29
 Assess the Evidence That the Researcher Used a
Good Instrument
• The scores from multiple questions were consistent with each
other (called internal consistency or Cronbach’s alpha, a). The
scores for all questions should relate to each other at a
positive, high level (Cronbach’s alpha = .7 - 1.0).
• ■ The scores from the instrument were consistent over time
when the instrument was administered more than once to the
same individuals (called test–retest reliability). The scores
from the two times should relate (or correlate) at a positive,
reasonably high level ( r = .6 - 1.0).
• ■ If multiple raters were used (such as with an observational
checklist), the different raters scored items in a consistent way
(called inter-rater reliability). Raters in quantitative research
should agree 90–100 percent of the time.
30
 Assess the Evidence That the Researcher Used a
Good Instrument
• When you read about the instruments used in a
quantitative study, you should also look for
information indicating that the scores from each
instrument are valid.
• Valid means that the scores from an instrument are
accurate indicators of the variable being measured
and enable the researcher to draw good
interpretations. That is, the scores should be useful
and meaningful measures of the variable of interest.

31
 Assess the Evidence That the Researcher Used a
Good Instrument
• You can identify evidence of valid scores by recognizing the
following kinds of information as you read:
• ■ The authors included a citation to the literature indicating
that the instrument was previously developed and used for
research purposes.
• ■ If the authors developed their own instrument, they included
the text of the items in the report so you can judge whether
they addressed the concept being measured in a clear and
unambiguous way.
• ■ The researchers used experts to assess that the items on the
instrument cover the appropriate content (called content or
face validity).
32
• Here is the paragraph about one of the
instruments in that study. As you read it,
identify the variable, instrument type, and what
evidence the researcher provided to indicate
that the study used a good instrument.

33
 Exercise
• Adolescent egocentrism was measured with the AES
[Adolescent Egocentrism Scale] developed by Enright and
colleagues (Enright et al., 1979; Enright et al., 1980). The
scale consists of three five-item subscales for the imaginary
audience, personal fable, and self-focus. For each item,
participants are asked to rate the importance of a statement on
a scale ranging from 1 = not important to 5 = very important.
The AES has demonstrated acceptable to good levels of
internal consistency ranging from Cronbach’s alpha = .78
(Enright et al., 1979) to Cronbach’s alpha = .83 (Enright et al.,
1980). Scores were calculated by summing across the five
items for each subscale. (Frankenberger, 2004, pp. 580 – 581 )
34
 Solution To Exercise
• Frankenberger clearly identified a major construct that she
wanted to measure—namely, adolescent egocentrism. She
specified three variables related to this construct: imaginary
audience, personal fable, and self-focus. She used an
instrument called the Adolescent Egocentrism Scale.
• In this paragraph, the author provided evidence that the scores
from this instrument are reliable (i.e., consistent) to an
“acceptable to good” level because Cronbach’s alpha is greater
than .7. She also provided limited evidence for the validity of
the scores by using an instrument that has been previously
reported in the literature

35