Sampling for Surveys

Surveys
What is a survey?

A process of presenting a standard series of questions to a
sample of persons.

The survey is the most widely used technique in criminology
because it is best suited for looking at the complex social world.
To capture that world accurately, we have to measure it
in situ.
That means taking information from selected people,
from where they are usually found.

Measures of many phenomena of interest are taken. The
purpose is to accurately reflect the beliefs, attitudes, and
behaviors of the sample in order to generalize accurate
information to a target population.


Surveys
Survey research typically uses sampling rather than taking
census.
Sampling vs. Taking a Census
Sampling: selecting cases (elements)or locating people (or other units of
analysis)from a target population in order to study the population.
Taking a Census: using all cases in an entire target (all elements) in order to
study the population
So why dont we always take a census?

A Sample is a:
Noun: the group from whom data are (or were) gathered, and
Verb: to select cases that represent a populationnot a musical term here

There are multiple ways to sample, but the goal is for the
sample to maximally represent the target population

Sample vs. Population
Population
Sample
Sampling
Types of Samples (Multiple units of analysis can be sampled):

Cases Persons in Field Studies Situations
Archival Data Experiment Participants
Persons answering a Survey

Depending on how the sample was generated, there are limits
to how much findings can be generalized from it.

One aims for broad generalizability, but type of sampling is
also determined by the:
complexities of the target population, and
researchers resources


Sampling
Sampling Techniques

Nonprobability: Sampling methods that do not let us
know in advance the likelihood of selecting for the
sample each element or case from a population
vs.
Probability: Sampling methods that allow us to know
in advance how likely it is that any element of a
population will be selected for the sample

Knowing the chance of selection allows one to control
sampling bias (under or overrepresentation of a
population characteristic in a sample)




Nonprobability Sampling
Nonprobability
(Very common in psychology, medicine, sociology)

1. Availability Sampling, convenience sampling
Selection of cases based on what is easiest to do
Experiments
Exploratory and Qualitative research
Avoid this if you can

2. Quota Sampling
Aspects of target population are known. Selects
availability sample ensuring that it reflects known
aspects of population

Nonprobability Sampling

3. Snowball Sampling
Respondent-driven sampling, initial respondents refer
others to the researcher
Usually used with hard-to-discover populations
Bias introduced by structured nature of affiliation
Can be improved with incentives to subjects to recruit a certain
number of new respondents

4. Purposive Sampling
Targets select people for a sample because of their
unique position
Helps get understanding of systems or processes or
information on a target population
Not representative of population in general



Nonprobability Sampling
Critiques

Limited generalizabilityone cannot judge representativeness.

Researchers should estimate who the sample represents . . . The
sample at least represents populations that are similar to it.

Why use nonprobability samples? Nonprobability does not mean,
intentional attempt to get a sample that is not representative:
1. Well-suited for exploratory and evaluation research
2. Sampling frames (lists from which samples are drawn) are at
times inadequate or nonexistent
3. Quick, efficient
4. Can be effectively used to study and describe social and social
psychological processes
5. Any research is limited, but not having research is worse.
6. Across samples, repeatedly finding the same results supports
generalizability.



Probability Sampling
Sampling Techniques
Probability Sampling: Sampling method reveals in advance the
likelihood that any one element will be selected for the sample

Probability sampling begins with a sampling frame, or a list of all
elements (or other units containing the elements) in a population.

E.g., Phone book, All Universities, Known Addresses, Subscribers to a
magazine.

If a sampling frame is incomplete (which they usually are) then the accuracy
of the sample is compromised. The researcher has the burden of assessing
the sampling error or bias.


Probability Sampling

1. Simple Random Sampling
Cases are identified strictly on the basis of chance.
Random number table to select from sampling frame
Random digit dialing
Equal probability of selection

2. Systematic Random Sampling
First case selected randomly from list, subsequent
cases are selected at equal intervals.
Typically the same as Simple Random Sampling
Be aware of periodicity

Probability Sampling
3. Cluster Sampling

Use when sampling frame is difficult to obtain, but
clusters are identifiable.

Randomly select clusters, then use obtainable
sampling frames within the clusters to select
cases.
Example: There is no national list of independent Baptists,
but almost all independent Baptist churches can be
identified. Members can be selected from membership lists.

Because clusters are generally homogeneous
(e.g., all white churches) it is better to maximize
the number of clusters and minimize number
of cases from each cluster

Probability Sampling
Multistage cluster sampling

Selecting clusters in two or more hierarchical
stages
(e.g., selecting states, then selecting churches,
then members)

Keep stages to a minimum because each stage
produces sampling error; more stages, more error
Probability Sampling

4. Stratified Random Sampling
Sampling frame divided into strata, cases drawn from
each stratum randomly.

Small subpopulations of interest may yield too few
cases in simple random sampling. To
compensate, the researcher draws samples from
each subpopulation independently.

Example: Latino population of Santa Clara County is
around 25%. A random sample of 100 would
produce 20 30 Latinostoo few to generalize to
Santa Clara County Latinos.

Probability Sampling
4. Stratified Random Sampling

Proportionate Stratified Sampling
Ensuring that population proportions are reflected in
proportions of each stratum of sample.
Population: 4% black, 25% Latino, 27% Asian, 44% white
Sample of 1,000: 40 black, 250 Latino, 270 Asian, 440 white

Disproportionate Stratified Sampling
Population proportions are NOT reflected in
proportions of each stratum of sample.
Population: 4% black, 25% Latino, 27% Asian, 44% white
Sample of 1,000: 250 black, 250 Latino, 250 Asian, 250 white
Idea is to get a lot of cases in each stratum
When combining all cases into one sample, use weighted
averages
2010 GSS Sampling
Full probability sample of US householdseach
household has an equal chance of being
selected

Used stratified area probability sampling
At the household level, 1 adult is selected at random
(Kish Table)

Sampling frame
Most cases came from a list of addresses from USPS (over 2/3)
Remaining cases from NORC-generated lists of households

2010 GSS Sampling
Stages used in four population area types, ending with random adult

1.Big MSAs (city), have USPS address42% of population
1. Primary sampling unit: tract (1 -2K Housing units)168 selected
2. Housing Unit Selected from USPS List

2.Intermediate MSAs or counties, have USPS address30% of population
1. Primary sampling unit: MSA or part of county30 selected
2. Secondary sampling unit: tract120 selected
3. Housing unit Selected from USPS List

3.Rural counties and Intermediate areas (2) without adequate USPS address list
25% of population
1. Primary sampling unit: County, all or part25 selected
2. Secondary sampling unit: Segment (constructed to contain 300 Housing Units)100 selected
3. Housing unit from NORC-listed master

4.Big MSAs (city), without adequate USPS address list3% of population
1. Primary sampling unit: Segment12 selected
2. Housing unit from NORC-listed master
2010 GSS Sampling
Source: http://www.fcsm.gov/03papers/keynotespeaker.pdf, January 12, 2012
Stratum 3 =
NORC list used
2010 GSS Sampling
What are the implications of the General Social
Surveys sampling???

The GSS is an adults-only survey of persons in
households. Therefore, it underrepresents:
18 24 year-olds (many not living in households
military, college, roaming)
65 and over (many not living in households
vacations, RVs, assisted living)
Persons who live in large households (only one
person per household is interviewed)
Homeless, criminal, and some poor (not in official
households, in shelters, on streets, in apartments)
Probability Sampling
Critiques
Just being random does not ensure that a sample is
representative or that the research is good.
Limited Sampling Frame
Think of presidential phone polls:
Who is at home? Type of person, day of polling, etc.
Who has a land line?

Problems of non-responserandom non-response okay,
but systematic non-response is biasing
Phone surveys typically do not report response rate. They
are often below 30%

How were questions worded: Measurement error

Problems of misspecified models: Leads to not asking the
right questions
Probability Sampling
Critiques
Is the Sample large enough?
Larger samples produce less sampling error
Too large is a waste of money
Big is good, but accurate and appropriate are better
Fraction of population sampled does not increase accuracy
unless fraction is very large

Larger samples are needed when:
The population is more heterogeneous.
There are more variables of interest.
The weaker the effects, or the smaller the differences
between groups,

TO SUM: MORE COMPLEXITY REQUIRES LARGER SAMPLES