Académique Documents
Professionnel Documents
Culture Documents
Chapter 14
DATA COLLECTION IN THE FIELD, NONRESPONSE ERROR,
AND QUESTIONNAIRE SCREENING
GENERAL CONCEPT MULTIPLE CHOICE QUESTIONS
1.
In an effort to increase the response rate in surveys, researchers sometimes have interviewers say "I
am not selling anything" at the beginning of the conversation. Some research on this has concluded
that including this statement:
a. does increase response rates by about 2 percent
b. does increase response rates by about 10 percent
c. has no effect on response rates
d. actually lowered response rates by about 2 percent
e. actually lowered response rates by about 10 percent
208
Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening
4.
Which of the following best represents what should be done about nonsampling error?
a. Because it cannot be measured, greater attempts must be made to measure it.
b. Because it cannot be measured, researchers must know the sources of nonsampling error and
learn how to minimize it.
c. Researchers must be aware that nonsampling error exists but because it exists only in a
theoretical sense, there are no known safeguards to minimize it.
d. Nonsampling error should be measured and controlled at all stages of the research process in
order to ensure a 100 percent accurate study.
e. Nonsampling error should be measured and controlled at all stages of the research process in
order to ensure a 95 percent accurate study.
A variety of nonsampling error occurs during data collection. These data collection errors may be
divided into:
a. fieldworker error and respondent error
b. data error in; data error out
c. intentional and fieldworker
d. unintentional and respondent error
e. directional and nondirectional nonsampling errors
209
Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening
8.
An interviewer gives the respondent very subtle cues such as saying "yes" to statements with which
the interviewer agrees. This is an example of:
a. "yes-saying"
b. unintentional respondent fraud
c. leading the respondent
d. interviewer misunderstanding
e. misunderstandings
Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening
13.
Marketing researchers in the UK studied different types of potential respondents and labeled them
as "Easy to Get" and "Hard to Get," among others. The findings of their research shows that "Hard
to Gets" are _________ and ________ than "Easy to Gets."
a. taller; weigh more
b. taller; are more talkative
c. younger; taller
d. younger; less likely to be employed
e. younger; more likely to be employed
An unintentional respondent error occurs when a respondent's interest in the survey wanes, and this
is known as:
a. ADD
b. attention loss
c. attention sampling error
d. misunderstanding
e. leading behavior
211
Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening
17.
When it comes to validation, an industry standard is to randomly select ________ of respondents for
call back.
a. 50 percent
b. 75 percent
c. 33.3 percent
d. 25 percent
e. 10 percent
212
Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening
22.
Which of the following is true regarding the role professional field data collection companies play
in terms of nonsampling error?
a. Thus far, these companies have failed to recognize nonsampling error.
b. Only a handful of these companies can be counted on to effectively control nonsampling error.
c. Although these companies give "lip service" to nonsampling error reduction, they offer little
help in the control of nonsampling error.
d. These firms are waiting on technological advances to help in the control of nonsampling error.
e. Because their existence relies on providing high quality data, field data collection firms are
relied on by researchers to know both the sources of and ways to control nonsampling error.
Which of the following is a data collection error that is unique to online surveys?
a. multiple submissions by the same respondent
b. fatigue
c. misunderstandings
d. cheating
e. distractions
Which of the following has been labeled the marketing research industry's biggest problem?
a. sampling error
b. falsehoods on the part of respondents
c. nonresponse error
d. item omissions
e. population misrepresentation
Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening
27. Which of the following best describes the trend in nonresponse rates in telephone surveys conducted
from 1979 to 2003?
a. response rates have continually increased
b. response rates have remained unchanged primarily due to the effects of online surveys
c. response rates have slowly declined over this period of years
d. response rates increased dramatically from 1979 to about 1988, and then saw a downward fall
that was just as dramatic
e. response rates increased dramatically from 1979 to about 1985, and then saw a downward fall
that was just as dramatic
Answer: (c) Difficulty: (Difficult) Page: 405
28.
A respondent reaches a certain point in the interview, and then decides not to answer any more
questions. This is known as:
a. quitting
b. nonresponse
c. item(s) omission
d. break-offs
e. failure to communicate
Of the factors that determine whether or not a person will participate in a survey, which of those
listed below was found by researchers to increase participation?
a. any topic dealing with sports
b. topics related to current TV shows that are also popular
c. topics relating to taxation and social welfare
d. topics related to youth
e. topics of interest to the potential respondent
214
Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening
31.
One decision rule used in the research industry relating to how to define a "completed survey" is to
define a completed survey as one in which all the:
a. primary questions have been answered
b. primary and secondary questions have been answered
c. primary, secondary, and tertiary questions have been answered
d. questions have been answered
e. classification questions have been answered
Callback attempts are important to reduce nonresponse and usually about ________ callback
attempts should be made to the not-at-homes, busy signals, and no answers.
a.
8 to 10
b.
7 to 10
c.
5 to 10
d.
3 to 10
e.
3 to 4
Which of the following methods used to reduce nonresponse error is best for reducing the number
of not-at-homes, busy signals, and no answers?
a. advance notification
b. monetary incentives
c. increasing the "wrong targets"
d. callbacks
e. decreasing the "ineligibles"
215
Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening
35.
Which of the following organizations has provided the industry with a standard for calculating
response rates?
a. American Marketing Association
b. International Marketing & Research Association (IMRA)
c. Council of American Survey Research Organizations (CASRO)
d. Marketing Research Association
e. International Council for Response Rate Standardization
216
Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening
39.
If, after conducting a survey, it is determined that nonresponse is a problem, one alternative that may
be considered is weighting. Essentially, weighting averages means:
a. applying weights (to accurately reflect the proportions of subgroups that represent the
population) to subgroup, means to compute an overall score that adjusts for the nonresponse of
that subgroup
b. applying weights to answers of subgroups until the answer is believed to accurately reflect the
way the true subgroup in the population would have answered the research question
c. reproportioning subgroups in the population in a way that matches how they are proportioned in
the sample, and then recalculating the survey data
d. applying a weighted value of .5 or less to all sample respondents' answers who are felt to be
underrepresented in the sample data
e. applying a weighted value of .5 or less to all sample respondents' answers who are felt to be
overrepresented in the sample data
When respondents have a tendency to respond favorably or unfavorably regardless of the question,
it is known as:
a. yea- or nay-saying
b. endzoning
c. unreliable response
d. research with low variability
e. None of the above; the respondent is obviously favorable or unfavorable toward the topic
being evaluated.
217
Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening
TRUE/FALSE QUESTIONS
43.
Some research has concluded that including the statement "I am not selling anything" has a small
beneficial effect on response rates but does not affect the percent of persons who terminate midinterview.
There are two kinds of error in surveys; sampling error and nonsampling error.
Several types of nonsampling error exist, but fortunately we can eliminate this error by having a
lower margin of allowable error (e).
We can predetermine (and control) the level of sampling error; we cannot predetermine the amount
of nonsampling error we will have.
Nonsampling errors stem primarily from either fieldworker errors or respondent errors.
One of the primary causes for interviewer cheating stems from the compensation method: payment
per completed interview.
Very subtle "cues" from the interviewer may lead the respondent to answer the questions correctly.
These "cues" should be properly taught to the interviewers during training so that the results may be
guaranteed (within limits).
218
Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening
51.
One characteristic of outstanding telephone interviewers is work orientation: a hard worker who
considers interviewing a personal skill.
Respondents can misunderstand, guess, lose attention, suffer distractions, and suffer respondent
fatigue. These are all examples of unintentional respondent error and they all contribute to
nonsampling error.
In research conducted to identify "hard to get" respondents and compare them with "easy to get"
respondents, "hard to gets" are typically older and less likely to be employed than "easy to gets."
Supervision and validation are ways the marketing research industry controls for interviewer
cheating.
Supervision and validation are ways the marketing research industry controls for interviewers
leading respondents.
An industry standard for validation is to recall 50 percent of the respondents to verify that they
were, in fact, interviewed.
Anonymity occurs when the respondent is given assurances that his or her answers will remain
private; the researcher will not divulge the name of the respondent with the responses.
Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening
60.
Reversal of scale endpoints is helpful in order to change the results to more closely match the
research objectives. If the objectives are met, the client will have greater satisfaction and will likely
use the services of the research firm again.
One of the problems with online surveys is that they may lead more people to submit bogus
responses about themselves, creating "bogus respondents."
One of the problems with online surveys is that, because many consumers are not connected to the
Internet, they may misrepresent the intended population.
Nonresponse is composed of refusal and item omission; break-offs are not considered a form of
nonresponse.
An item omission refers to the failure to include an important question on a survey questionnaire.
Therefore, item omissions may seriously threaten the achievement of the research objectives.
It was long thought that response rates increased if the respondents have a high level of interest in
the research topic. However, recent research has shown this is false.
A completed interview is always defined as an interview in which 100 percent of the questions have
been answered and in a way that they are all supportive of the research objectives.
220
Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening
68.
A good definition for a completed interview is one that has all questions answered up to the
demographic questions at the end of the survey.
If a respondent is called and is "not at home," that sample element is no longer usable and callbacks
should not be made. To do otherwise will distort the response rate.
If a respondent is not at home or if their telephone is busy, it is standard practice in the research
industry to immediately substitute that sample member with the next person on the sample frame.
CASRO, the Council of American Survey Research Organizations, has provided standards for
calculating response rates.
Response rate and nonresponse rates are related; if the response rate is 60 percent, then the
nonresponse rate is 40 percent.
Response rate is calculated as the number of units in the sample divided by the number of
completed interviews.
In calculating the response rate for studies with screening questions and, therefore, a certain
percentage of the sample who are "eligible," it is impossible to estimate the percentage of those
members of the sample who are not called (i.e. not at homes, busy signals, etc.) who are "eligible."
Therefore, the response rate formula omits the concept of eligibility, thus giving us response rates
much lower than they should be.
221
Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening
76.
If a study is conducted and nonresponse error is a problem, one remedy may be applying weighted
averages that essentially increase or decrease a subgroup's proportion in the sample to that
proportion known to exist in the population.
One remedy for studies that are thought likely to have high nonresponse rates is to oversample,
which means we draw a sample larger than calculated in order to meet the required sample size
calculated by the sample size formula.
When a researcher sees a "middle-of-the-road pattern" during the questionnaire screening process, it
signals that a good, average response was obtained by the survey and this is a good sign that the
research process did not produce strange abnormalities.
Sometimes a researcher will include an internal consistency check to ensure that reliable answers
are being given by respondents. This is normally carried out by having two different questions that
measure the same construct imbedded somewhere in the questionnaire.
Rachel Luecht was responsible for conducting a survey of her university to determine what extent
the students were in favor of a number of changes the university administration was considering.
She would need to collect four hundred completed interviews in a five-day period, and the
administration had given her a budget of $2,000.00 to collect the information. Rachel put posters up
around the campus asking for students who wanted to earn extra cash by interviewing other students
on campus. At the meeting of the volunteers, Rachel offers to pay them $8.00 an hour and told them
they should complete at least three interviews for each hour they worked. Rachel's decision to pay
the interviewers this way will:
a. help ensure there are no misunderstandings among the fieldworkers collecting the information
b. help ensure that there is a minimum of fatigue among the fieldworkers collecting the
information
c. help ensure that her fieldworkers do not cheat
d. help ensure that leading respondents will be held at a minimum because she is giving the
fieldworkers ample time to conduct the interviews
e. help control respondent misunderstandings
222
Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening
81.
Margaret Day Craddock is the director of MIFA, a charitable organization in Memphis, Tennessee.
Margaret has contracted with a marketing research firm in Memphis to conduct a telephone survey
of Memphis MSA residents to assess their awareness of charitable organizations, tax code
provisions for charitable donations, attitudes toward charitable organizations, and intentions to
contribute in the coming year. Not everyone contacted will be eligible to take part in the study as
there is a screening question which asks if the potential respondent is the person in the household
that is most likely to make decisions regarding the family's contributions to charity. Because
Margaret plans on using the research information to help her set objectives and the budget for the
coming year, she has instructed the research firm to ensure that the information is representative of
the MSA. Bob Smith, of the research firm, is using a probability sample and he is taking
safeguards to minimize nonsampling error. Today, Bob is working on factors that will affect the
response rate. Because there will be a screening question, Bob knows that this will:
a. not have any impact on the response rate
b. not have any impact on the nonresponse rate
c. have an impact on the response rate in that only those respondents who are "eligible" (i.e. pass
the screen), will be included in the denominator of the response rate formula
d. have an impact on the response rate in that only those respondents who are "ineligible" (i.e. fail
to pass the screen), will be included in the denominator of the response rate formula
e. lower the response rate because many persons contacted will not be responsible for making
contribution decisions
Bob Smith is conducting a survey for Margaret Day Craddock, the director of MIFA, a charitable
organization in Memphis, Tennessee. Margaret has contracted with Bob's marketing research firm
in Memphis to conduct a telephone survey of Memphis MSA residents. Not everyone contacted
will be eligible to take part in the study as there is a screening question which asks if the potential
respondent is the person in the household that is most likely to make decisions regarding the
family's contributions to charity. Bob will use the CASRO formula, discussed in your text, to
determine the response rate for the study. The formula essentially divides the number of completed
interviews by the number of eligible units (respondents) in the sample. Which of the following best
represents how the number of eligible units in the sample is calculated?
a. 100 percent of the samplethose failing the screening question
b. 100 percent of the sample; ALL respondents are "eligible" whether they pass the screen or not
c. ("eligibles" consist of those passing the screen) plus (the percentage of those eligible among
respondents we talked with) times (the number of respondents we did not talk with due to
refusals [before the screen], no answers, busy signals, or not at homes)
d. ("ineligibles" consist of those passing the screen) plus (the percentage of those ineligible
among respondents we talked with) times (the number of respondents we did not talk with due
to refusals [before the screen], no answers, busy signals, or not at homes)
e. None of the abovebecause there is no CASRO formula
223