An Introduction to Survey

Research, Volume II
An Introduction to Survey
Research, Volume II
Carrying Out the Survey

Second Edition

Ernest L. Cowles
Edward Nelson
An Introduction to Survey Research, Volume II, Second Edition: Carrying
Out the Survey
Copyright © Business Expert Press, LLC, 2019.

All rights reserved. No part of this publication may be reproduced, stored

in a retrieval system, or transmitted in any form or by any means—
electronic, mechanical, photocopy, recording, or any other except for
brief quotations, not to exceed 250 words, without the prior permission
of the publisher.

First published in 2019 by

Business Expert Press, LLC
222 East 46th Street, New York, NY 10017
www.businessexpertpress.com

ISBN-13: 978-1-94999-128-4 (paperback)

ISBN-13: 978-1-94999-129-1 (e-book)

Business Expert Press Quantitative Approaches to Decision Making

Collection

Collection ISSN: 2153-9515 (print)

Collection ISSN: 2163-9582 (electronic)

Cover and interior design by S4Carlisle Publishing Services Private Ltd.,

Chennai, India

First edition: 2015

Second Edition: 2019

10 9 8 7 6 5 4 3 2 1

Printed in the United States of America.

 Dedication
As we complete this second edition of Introduction to Survey Research, I am
again indebted to the many people who have provided me with support,
insights, and patience along the way. My wife, Ellison, deserves a special
thank-you for having given up many opportunities for adventures because
I was sequestered at my computer for “just a little while longer.” I am also
grateful for her help in editing, reference checking, and other commonplace
aspects of writing the book, which plunged her deeper into survey research
than she probably would have liked. Finally, I am very fortunate to have
had Ed Nelson working with me, who, despite suffering a deep personal loss,
persevered in completing the manuscript. His work and dedication continue
to make him an invaluable colleague.

—Ernest L. Cowles
I dedicate this book to my wife, Elizabeth Nelson, and my children, Lisa
and David, for all their support over many years. Elizabeth and I were
both in the Sociology Department at California State University, Fresno,
for many years and shared so much both at work and at home with our
family. It has been a pleasure to work with my coauthor, Ernest Cowles,
on this book. Both of us were directors of survey research centers until our
retirements, and we have combined our years of experience in this project.

—Edward Nelson
 Abstract
This two-volume work updates a previous edition of our book that was
­intended to introduce the fundamentals of good survey research to ­students
and practitioners of the survey process as well as end users of survey informa-
tion. It describes key survey components needed to d ­ esign, understand, and
use surveys effectively and avoid the pitfalls stemming from bad survey con-
struction and inappropriate methods. In the first volume, we first considered
the ways to best identify the information needed and how to structure the
best approach to getting that information. We then reviewed the processes
commonly involved in conducting a survey, such as the benefit of repre-
sentative sampling and the necessity of dealing with the types of errors that
commonly distort results. Volume I concluded with a chapter that examines
the elements to consider when developing a survey followed by a chapter
that acquaints the reader with the different modes of delivering a survey.
In this second volume, we focus on carrying out a survey. We begin
with a brief overview of the importance of research questions and the
research design and go on to discuss key elements in writing good ques-
tions. We then focus on the steps that researchers must go through when
conducting the survey. We next turn our attention to the impacts of rap-
idly changing technology on survey research, including rapidly evolving
mobile communication, specifically, online access, the expanded use of
web-based surveys, and the use of survey panels, and the opportunities
and challenges provided by access to “Big Data.” We ­conclude with an
important, but often neglected, aspect of surveys: the presentation of re-
sults in different formats appropriate to different audiences. As with the
previous edition, each chapter concludes with a summary of important
points contained in it and an annotated set of references for readers who
want more information on chapter topics.

Keywords
Big Data; ethical issues; Internet surveys; interviewer training; mailed
­surveys; mixed-mode surveys; mobile devices; sampling; surveys; s­urvey
content; survey construction; survey panels; survey processes; survey
­reports; survey technology; telephone surveys; web surveys
 Contents
Preface...................................................................................................xi
Acknowledgments..................................................................................xiii

Chapter 1 Introduction......................................................................1
Chapter 2 Writing Good Questions....................................................7
Chapter 3 Carrying Out the Survey..................................................33
Chapter 4 Changing Technology and Survey Research......................53
Chapter 5 Presenting Survey Results.................................................73

Notes....................................................................................................99
References............................................................................................107
About the Author.................................................................................117
Index..................................................................................................119
 Preface
Survey research is a widely used data collection method that involves
­getting information from people, typically by asking them questions and
collecting and analyzing the answers. Such data can then be used to un-
derstand individuals’ views, attitudes, and behaviors in a variety of areas,
including political issues, quality of life at both the community and the
individual levels, and satisfaction with services and products. Decision
makers in both the public and the private sectors use survey results to
understand past efforts and guide future direction. Yet there are many
misperceptions about what is required to conduct a good survey. Poorly
conceived, designed, and executed surveys often produce results that are
meaningless, at best, and misleading or inaccurate, at worst. The resultant
costs in both economic and human terms are enormous.
Our purpose of writing this two-volume edition is to provide an in-
troduction to and overview of survey research. In Volume I, we intro-
duced key elements of information gathering, specifically, identifying the
information needed and the best way to get that information. We then
explored the importance of representative sampling and identifying and
minimizing survey errors that can distort results. The remainder of the
first volume was focused on the practical issues to consider when develop-
ing, building, and carrying out a survey, including the various modes that
can be used to deliver a survey.
This volume (Volume II) focuses on carrying out the survey. We
­introduce survey implementation by first looking at the importance of
research questions in the research design. We also discuss key elements
in writing good questions for various types of surveys. We next take the
reader through the key steps that researchers must go through when con-
ducting the survey. We then highlight some of the major factors that
are influencing the way today’s surveys are being delivered such as the
­rapidly changing technology that is rapidly transforming the construction
and presentation of surveys. The rapidly changing landscape of mobile
xii PREFACE

communication, including online access to web-based surveys, the cre-

ation of survey panels, and the use of “Big Data,” presents exciting new
opportunities for survey research, but also new obstacles and challenges
to its validity and reliability. We conclude with an important, but often
neglected, chapter dealing with the presentation of results in different
formats appropriate to different audiences. As with the previous edition,
each chapter concludes with a summary of important points contained
in the chapter and an annotated set of references indicating where readers
can find more information on chapter topics.
 Acknowledgments
We would like to thank the S4Carlisle Publishing Services team for
their great work and contributions which have significantly improved
this work. We also acknowledge the help and support of Scott Isenberg,
the Executive Acquisitions Editor of B ­ usiness Expert Press, and Donald
­Stengel, the Collection Editor for a group of their books under a collec-
tion called Quantitative Approaches for Decision Making. Don also read
our manuscript and offered useful and valuable suggestions.
 CHAPTER 1

Introduction

Research starts with a question. Sometimes these are why questions. Why
do some people vote Democrat and others vote Republican? Why do
some people purchase health insurance and others do not? Why do some
people buy a particular product and others buy different products? Why
do some people favor same-sex marriage and others oppose it? Why do
some people go to college and others do not? Other times they are how
questions. If you are a campaign manager, how can you get people to vote
for your candidate? How could we get more people to purchase health
insurance? How could you get customers to buy your product? How
could we convince more people to go to college? But regardless, research
starts with a question.
Have you thought about how we go about answering questions in
everyday life? Sometimes we rely on what people in authority tell us.
Other times we rely on tradition. Sometimes we use what we think is
our common sense. And still other times we rely on what our gut tells
us. But another way we try to answer questions is to use the scientific
approach.
Duane Monette et al. suggest that one of the characteristics of the
scientific approach is that science relies on systematic observations.1 We
often call these observations data and say that science is empirical. That
means it is data based. However, the scientific approach doesn’t help you
answer every question. For example, you might ask whether there is a
God, or you might ask whether the death penalty is right or wrong. These
types of questions can’t be answered empirically. But if you want to know
why some people vote Democrat and others vote Republican, the sci-
entific method is clearly the best approach. Relying on what people in
authority tell you or what tradition tells you or your gut won’t work.
2 AN INTRODUCTION TO SURVEY RESEARCH, VOLUME II

Research Design
Your research design is your plan of action. It’s how you plan to answer
your research questions. Ben Jann and Thomas Hinz recognize the im-
portance of questions when they say that “surveys can generally be used
to study various types of research questions in the social sciences.”2 The
research design consists of four main parts—measurement, sampling,
data collection, and data analysis. Measurement is about how you will
measure each of the variables in your study. Sampling refers to how you
will select the cases for your study. Data collection is about how you
plan to collect the information that you will need to answer the research
questions. And data analysis is about how you plan to analyze the data.
You need to be careful to decide on your research design before you col-
lect your data.
In this book, we’re going to focus on data collection, specifically on
surveys. The book is organized in two volumes. In the first volume we
focused on the basics of doing surveys and talked about sampling, survey
error, factors to consider when planning a survey, and the different types
of surveys you might use. In the second volume we’ll focus on carrying
out the survey, and we’ll discuss writing good questions, the actual carry-
ing out of surveys, the impacts of current technology on survey research,
and survey reporting.

Questioning (Interviewing) as a Social Process

Surveys involve asking people questions. Usually, this is referred to as in-
terviewing, which is in some ways similar to the types of conversations we
engage in daily but in other ways very different. For example, the inter-
viewer takes the lead in asking the questions, and the respondent has little
opportunity to ask the interviewer questions. Once the respondent has
consented to be interviewed, the interviewer has more control over the
process than does the respondent. However, it is the respondent who has
control over the consent process, and it is the respondent who determines
whether and when to terminate the interview. We discussed nonresponse
in Chapter 3 (Volume I), “Total Survey Error,” and we’ll discuss it further
in Chapter 3 (Volume II), “Carrying Out the Survey.”
 INTRODUCTION
3

Raymond Gorden has provided a useful framework for viewing the

interview as a social process involving communication. Gordon says that
this communication process depends on three factors: “the interviewer,
the respondent, and the questions asked.”3 For example, the race and gen-
der of the interviewer relative to that of the respondent can influence
what people tell us, and we know that the wording and order of questions
can also influence what people tell us. We discussed these considerations
in Chapter 3 (Volume I), “Total Survey Error.”
Gorden goes on to suggest that the interaction of interviewer, respon-
dent, and questions exists within the context of the interview situation.4
For example, are we interviewing people one-on-one or in a group set-
ting? Many job interviews occur in a one-on-one setting, but one of the
authors recalls a time when he was among several job applicants who
were interviewed in a group setting involving other applicants. Rest as-
sured that this affected him and the other applicants. Are we interviewing
people in their homes or in another setting? Think of what happens in
court when witnesses are questioned in a courtroom setting. That clearly
affects their comfort level and what they say.
Gorden notes that the interview and the interview situation exist
within the context of the culture, the society, and the community.5 There
may be certain topics, such as religion and sexual behavior, that are diffi-
cult to talk about in certain cultures. Norms of reciprocity may vary from
culture to culture. Occupational subcultures, for example, the subcultures
of computer programmers and lawyers, often have their own language.
Norman Bradburn views “the interview as a microsocial system con-
sisting of two roles, that of the interviewer and that of the respondent.
The actors engage in an interaction around a common task, that of com-
pleting an interview.”6 He goes on to suggest that “there are general social
norms that govern interactions between strangers.”7 Two of those norms
are mutual respect (including privacy) and truthfulness.
It’s helpful to keep in mind that the interview can be viewed as a social
setting that is affected by other factors, as is the case with any other social
setting. In this book, we will be looking at many of the factors that affect
the interview. We’ll look at the research that has been done and how we
can use this research to better conduct our interviews.
4 AN INTRODUCTION TO SURVEY RESEARCH, VOLUME II

Book Overview
Here’s a brief summary of what we covered in the first volume of this book.

• Chapter 1—Introduction—Interviewing isn’t the only way we can

obtain information about the world around us. We can also ob-
serve behavior. We compared observation and interviewing as two
different methods of data collection. We also looked at a brief his-
tory of social surveys.
• Chapter 2—Sampling—What are samples and why are they used?
In this chapter, we discussed why we use sampling in survey re-
search, and why probability sampling is so important. Common
types of samples are discussed along with information on choosing
the correct sample size and survey approach.
• Chapter 3—Total Survey Error—Error is inevitable in every scien-
tific study. We discussed the four types of survey error—sampling,
coverage, nonresponse, and measurement error, focusing on how
we can best minimize it.
• Chapter 4—Factors to Consider When Thinking about Surveys—
In this chapter some of the fundamental considerations about sur-
veys were presented: the stakeholders and their roles in the survey
process; ethical issues that impact surveys; factors that determine
the scope of the survey; and how the scope, in turn, impacts the
time, effort, and cost of doing a survey.
• Chapter 5—Modes of Survey Delivery—There are four basic
modes of survey delivery—face-to-face, mailed, telephone, and
web delivery. We focused on the critical differences among these
different modes of delivery and the relative advantages and disad-
vantages of each. We also discussed mixed-mode surveys, which
combine two or more of these delivery modes.

Here’s what we are going to discuss in this second volume.

• Chapter 2—Writing Good Questions—Here we look at survey

questions from the perspective of the researchers and the survey
 INTRODUCTION
5

participants. We focus on the fundamentals of the design, format-

ting, and wording of open- and closed-ended questions and discuss
some of the most commonly used formats in survey instruments.
• Chapter 3—Carrying Out the Survey—Every survey goes through
different stages, including developing the survey, pretesting the
survey, administering the survey, processing and analyzing the
data, reporting the results, and making the data available to others.
Surveys administered by an interviewer must also pay particular
attention to interviewer training.
• Chapter 4—Changing Technology and Survey Research—The
chapter focuses on the impacts technology has had on survey
research. As computers became widely available to the general
public, survey platforms adapted to self-administered formats.
Likewise, as cell-phone technology replaced landline telephones,
survey researchers had to adapt to new issues in sampling method-
ology. Currently, rapid advances in mobile technology drive both
opportunities and challenges to those conducting surveys.
• Chapter 5—Presenting Survey Results—In this chapter we talk
about the last step in the survey process—presenting the survey
findings. Three major areas, the audience, content, and expression
(how we present the survey), which shape the style and format of
the presentation, are each discussed along with their importance
in the creation of the presentation. The chapter concludes with a
discussion on how different types of presentations such as reports,
executive summaries, and PowerPoints can be structured and how
survey data and results can be effectively presented.

Annotated Bibliography
Research Design

• Matilda White Riley’s Sociological Research I: A Case Approach is an

early but excellent discussion of research design.8 Her paradigm of
the 12 decisions that must be made in constructing a research design
includes the alternative methods of collecting data—observation,
questioning, and the combined use of observation and questioning.
6 AN INTRODUCTION TO SURVEY RESEARCH, VOLUME II

• Earl Babbie’s The Practice of Social Research is a more recent intro-

duction to the process of constructing a research design.9
• Delbert Miller and Neil Salkind’s Handbook of Research Design &
Social Measurement provides many examples of the components of
the research design.10

Questioning (Interviewing) as a Social Process

• Raymond Gorden’s Interviewing: Strategy, Techniques, and Tactics is

one of the clearest discussions of the communication process and
the factors that affect this process.11
• Norman Bradburn’s article “Surveys as Social Interactions” is an
excellent discussion of the interactions that occur in interviews.12
 Index
AAPOR Task Force Report, 66 Data, 1
Active listening, 44 coding, 42
American Association for Public collection, 2
Opinion Researchers confidentiality agreements, 83
(AAPOR), 83 editing, 42–43
American Consumer Opinion, 61 making available to other social
American Trends Panel (ATP), 60 scientists, 43–44
Answering questions, willingness to, presenting, 85–86
13–14 Data analysis, 2, 43
Audience, 74–75 Data entry, 43
knowing, 75 Deliverables, 74–75
lack of understanding by, 75–77 Digital divide, 58
types of, 77–79 Disposition codes, 40
Documentation, for interviewer,
Behavior coding, 35 45–46
Big Data, 64–66 Double-barrel questions, 16
Breakoff rates, 58–59 Double-negative questions, 17
Brevity, question, 14–15 Dynamic probes, 39
Built-in assumptions questions, 17
Editing, data, 42–43
Call record, 40–41 Empirical, 1
Callback, 40 Ethical issues, 66–68
Changing technology, and survey Executive summaries, 81–82
research, 53–71 Experts, reviewing survey, 35
Chartjunk, 90 Expression, survey results, 85
Checks-and-balances, 85 presenting data, 85–86
City of residence, 11–12 text, graphs, and tables, 86–93
Clarity, question, 14–15
Closed-ended questions, 20–28 Face-to-face survey, 36, 39
Coding, 42 Focus groups, 34
behavior, 35
Cognitive interviews, 34 Good questions. See also Survey
Computer-assisted personal questions
interviewing (CAPI), 55 key elements of
Computer-assisted telephone common question pitfalls,
interviewing (CATI), 9, 55 avoiding, 16–17
Confidentiality, 83–84 specificity, clarity, and brevity,
Content, survey results, 79–81 14–15
executive summaries, 81–82 writing, 7–31
privacy, confidentiality, and goals for, 11
proprietary information, Google, 34
83–84 Google Scholar, 34
120 INDEX

Granularity, 24 New York Times, 53

Graphs, 86–93 Nonresponse bias, 47
Nontechnical audience, 77–78
Human subject protections, 83
Online panels, 60–63
IBM RAMAC 305, 54 Online survey, 9, 19, 57, 61–62
Inflation, 8 Open-ended questions, 18–20
Inter-university Consortium for Opt-in panels, 60–64
Political and Social Research Ordered/scalar response category, 25
(ICPSR), 43–44
Interactive probes, 39 Participants, 74
Internet, 64 Pew Research Center, 56–57
access, 56–59 Practice interviews, 46
-accessible smartphones, 57 Pretesting, 35–37
Interviewer Privacy, 83–84
-administered survey, 44 Probability-based panels, 59–60
debriefing, 35 Probability surveys, 64–66
manual, 46 Probes, defined, 38
–respondent–questions interaction, Probing
3 in mailed surveys, 39–40
training in web surveys, 39
practice interviews, 46 Proprietary information, 83–84
providing documentation for,
45–46 Questioning, as social process,
Interviewing. See Questioning 2–3, 6
Questions, 1–6
Level of granularity, 80
Likert Scale, 25–26 Record keeping, 40–41
Listening, 44 Refusals, 45
Loaded/leading questions, 16 Reliability, 11–13
Research design, 2
Mailed surveys, probing in, 39–40 Researchers, 74
Marketing Research, 73 Response rate, 41, 47
MCAPI, 55–56 Role playing, 46
MCATI, 55–56
Measurement, 2 Sampling, 2, 7
Mixed-mode surveys, 8–9 Satisficing, 36
advantages of, 10 Scientific approach, 1
Mobile technologies, 64. See also Self-administered survey, 41
MCAPI; MCATI Semantic Differential Scale, 26–28
Modern computing, 53–55 Skip patterns, 36
internet access and survey mobility, Smartphones
56–59 dependent, 57
wireless telephones, 55–56 internet-accessible, 57
Moore’s Law, 54 Social process, questioning as, 2–3, 6
Specificity, question, 14–15
Narrative presentation. See Text Sponsors, 74
National Center for Health Statistics, 56 Statistical story-telling, 86
 INDEX
121
Structured questions. See Closed-
ended questions
Survey mobility, 56–59
Survey panels, 59
opt-in panels, 60–64
probability-based panels, 59–60
Survey presentations, 73
Survey questions, 7–31
good questions
key elements of, 14–17
writing, 7–31
responses to, 87
types and formats, 17–18
closed-ended, 20–28
open-ended, 18–20
validity and reliability in, 11–13
willingness to answer, 13–14
Survey reports
format of, 79–80
writing, 43
Survey results
audience, 74–75
knowing, 75
lack of understanding by, 75–77
types of, 77–79
content, 79–81
executive summaries, 81–82
privacy, confidentiality, and
proprietary information,
83–84
expression, 85
presenting data, 85–86
text, graphs, and tables, 86–93
presenting, 73–97
Survey technology, 53–71. See also
specific technologies
SurveyMonkey, 61–62
Surveys, 33–51
administering
linking to other information,
41–42
probe questions, 37–39
probing in mailed surveys, 39–40
probing in web surveys, 39
record keeping, 40–41
behavior coding and interviewer
debriefing, 35
developing, 7, 33–34
cognitive interviews, 34
focus groups, 34
other surveys, 34
interviewer training
practice interviews, 46
providing documentation for,
45–46
listening, 44
making data available to other social
scientists, 43–44
mode of delivery, 37
participation, 47–48
pretesting, 35–37
processing data
coding, 42
data analysis, 43
data entry, 43
editing, 42–43
writing reports, 43
Tables, 86–93
Tailored Design Method, 9
Technical audience, 77–78
Telephone survey, 35, 36, 39
disposition codes for, 40
Telephones, wireless, 55–56
Text, 86–93
Total Design Method, 9
UNIVAC, 53–54
Universal Automatic Computer. See
UNIVAC
Unstructured questions. See Open-
ended questions
Validity, 11–13
Wall Street Journal, 54
Web surveys, probing in, 39
Wireless telephones, 55–56
YouGov, 61
 OTHER TITLES IN QUANTITATIVE APPROACHES
TO DECISION MAKING COLLECTION
Donald N. Stengel, California State University, Fresno, Editor
• Regression Analysis: Understanding and Building Business and Economic Models Using
Excel, Second Edition by J. Holton Wilson, Barry P. Keating, and Mary Beal-Hodges
• Operations Methods: Managing Waiting Line Applications, Second Edition
by Kenneth A. Shaw
• Using Statistics for Better Business Decisions by Justin Bateh and Bert G. Wachsmuth
• Applied Regression and Modeling: A Computer Integrated Approach by Amar Sahay
• The Art of Computer Modeling for Business Analytics: Paradigms and Case Studies
by Gerald Feigin
• Data Visualization, Volume I: Recent Trends and Applications Using Conventional
and Big Data by Amar Sahay
• Data Visualization, Volume II: Uncovering the Hidden Pattern in Data Using Basic
and New Quality Tools by Amar Sahay
• Decision Analysis for Managers, Second Edition: A Guide for Making Better Personal
and Business Decisions by David Charlesworth
• Conducting Survey Research: A Practical Guide by John Fogli and Linda Herkenhoff
• MS Excel, Second Edition: Let’s Advance to the Next Level by Anurag Singal
• Business Decision Making, Second Edition: Streamlining the Process for More Effective
Results by Milan Frankl
• An Introduction to Survey Research, Second Edition, Volume I: The Basics of Survey
Research by Ernest L. Cowles and Edward Nelson

Announcing the Business Expert Press Digital Library

Concise e-books business students need for classroom and research

This book can also be purchased in an e-book collection by your library as

• a one-time purchase,
• that is owned forever,
• allows for simultaneous readers,
• has no restrictions on printing, and
• can be downloaded as PDFs from within the library community.

Our digital library collections are a great solution to beat the rising cost of textbooks. E-books
can be loaded into their course management systems or onto students’ e-book readers.

The Business Expert Press digital libraries are very affordable, with no obligation to buy in
future years. For more information, please visit www.businessexpertpress.com/librarians.
To set up a trial in the United States, please email sales@businessexpertpress.com