CHAPTER 5: CONSTRUCTING

OPEN- AND CLOSED-ENDED

QUESTIONS
Damon Burton
University of Idaho

What is the difference

between open- and
closed-ended
questions?

QUESTION-TYPE
DEFINITIONS

Open-Ended

Questions are questions

in which the choice of responses are
not specified and respondents must
generate their own responses.
Close-Ended Questions are items in
which responses choices are specified
and respondents must choose one of
the available alternatives.

TYPES OF OPEN-ENDED
QUESTIONS

Descriptive Questions in which respondents are

asked to provide in-depth information on the
topic of the question.
Numerical Response Questions respondents must
provide numerical data such as dates, frequency,
monetary value, count, amount or scalar value.
List-of-Items Questions respondents must
provide a list such as grocery stores they
frequent, brands of clothing, or
major
purchases theyd like to make.

Were going to discuss a

number of different
question types and show
you how to best write
each type of question.

NUMERICAL RESPONSE
GUIDELINES
1.Ask

for the specific unit desired in the

question stem (5.1).
2.Provide answer spaces that are sized
appropriately for the response task (5.2).
3.Provide unit labels with the answer
spaces (5.1).
4.Specificity enhances response accuracy.

5.1

NUMBER BOX QUESTIONS

5.2 IMPACT OF ANSWER

SPACES ON RESPONSES

DESCRIPTION &
ELABORATION GUIDELINES

1.Provide
2.Provide

extra motivation to respond (5.3).

adequate space for respondents to
completely answer the question (5.3).
3.Use scrollable boxes on web surveys
(5.3).
4.Consider programming probes to openended responses in internet surveys (5.4).

5.3 DESCRIPTION &

ELABORATION QUESTIONS

= 4/DAY

5.4 PROBING OPEN-ENDED

QUESTIONS

Youll probably use more

closed- than open-ended
questions, so here are
guidelines for how to write
good items.

CLOSED-ENDED
GUIDELINES

1.State

both positive and negative

sides in the question stem when asking
either/or questions (5.5).
2.Develop lists of answer categories
that include all reasonable possible
answers (5.6).
3.Develop lists of answer categories
that are mutually exclusive
questions (5.7).

5.5 INCLUDE POSITIVE

& NEGATIVE SIDES

= 4/DAY

5.6 MUTUALLY
EXCLUSIVE CATEGORIES

= 4/DAY

5.7 ORDERING OF
RESPONSES

= 4/DAY

CLOSED-ENDED
GUIDELINES
4. Maintain spacing between answer categories

that is consistent with measurement intent

(Figure 5.8).
5. Ask respondents to rank only a few items at
once rather than a long list (Figure 5.9).

Ranking questions are difficult to understand

and complete correctly.
Only have respondents rank top 3-4.
Pair all possible options and calculate overall
rankings during analysis.

5.8 SPACING RESPONSE

OPTIONS EVENLY

= 4/DAY

5.9 CLOSED-ENDED WITH

UNORDERED RESPONSES

= 4/DAY

CLOSED-ENDED
GUIDELINES
6. Avoid bias from unequal comparisons
(Figure 5.10).
7. Randomize response options if there
is concern about order effects (Figure
5.11).
8.Use forced-choice questions instead
of check-all-that-apply questions
(Figure 5.11).

5.10 AVOIDING UNEQUAL

COMPARISON BIAS

= 4/DAY

5.11 UNORDERED
RESPONSE OPTIONS

= 4/DAY

5.12 COMPARISON OF
RESPONSE RATES

= 4/DAY

CLOSED-ENDED
GUIDELINES
9. Use differently shaped answer spaces
(circles vs squares) to help
respondents distinguish between
single- and multiple-answer questions
(Figure 5.13).

5.13 QUESTION TYPE &

SHAPES OF ANSWER SPACES

= 4/DAY

How do the type of

analyses you plan to do
influence your
response choices?

ORDINAL CLOSEDENDED GUIDELINES

10. Choose appropriate scale length of
4-5 categories (Figure 5.14).

Typically you must have a minimum of 5

choices on a Likert scale to assume interval
data and utilize inferential statistics.
Even versus odd-number scales are based on
whether you want to allow respondents
to
choose a neutral response.

5.14 SCALER QUESTIONS

AND RESPONSE OPTIONS

= 4/DAY

ORDINAL CLOSEDENDED GUIDELINES

11. Choose direct or construct-specific labels
to improve understanding
(Figure 5.15).
12. Provide scales that approximate the
actual distribution of the
characteristic
in the population.

Typically respondents assume that the midpoint

response category represents the middle of the
distribution in the
population.

How does the reason for

conducting the survey
influence response options
(e.g., academic research
versus consumer
satisfaction)?

ORDINAL CLOSEDENDED GUIDELINES

13. Provide balanced scales where categories are
relatively equal distance apart conceptually
(Figure
5.16).
14. Consider how verbally labeling and visually
displaying
all response categories may
influence answers (Figure
5.17).

Dillman suggests that most surveyors prefer fully labeled

scales that range from 2-5 choices.
Conversely, researchers tend to prefer numerical
scales of 5-9-point Likert scales (see examples).

5.16 EVENLY-SPACE
CONCEPTUAL CATEGORIES

= 4/DAY

5.17 VERBAL VERSUS

NUMERICAL LABELS

= 4/DAY

ORDINAL CLOSEDENDED GUIDELINES

15.

Carefully evaluate the use of

numeric
labels and their impact on
measurement
(see examples).

Numbers reinforce the interval nature of data.

Dillman contends that numbers add to processing
time, but this depends on whether questions use
same response categories or change categories
from question to question.

Number consistently across questions.

ORDINAL CLOSEDENDED GUIDELINES

16. Align response options vertically in one
column or horizontally in one row and strive
for equal distance between categories (Figure
15.18).
17. Place nonsubstantive options at the end of
the scale and separate from
substantitive
options (Figure 15.19).

Be very careful about using dont know or no

opinion categories.