Questionnaire Design

Learning Objectives
Upon completion of this chapter, you will be able to:
 Understand the meaning and importance of questionnaire
 Understand three stages of the questionnaire design process
 Learn about pre-construction phase of the questionnaire
 Learn about construction phase of the questionnaire
 Learn about post-construction phase of the questionnaire
 What is a Questionnaire?
• A questionnaire consists of formalized and pre-specified
set of questions designed to obtain responses from
potential respondents.
• Questions in the questionnaire reflect the research
objective under investigation.
• Questionnaires are generally situation and culture
specific.
• Questionnaire design process requires a careful attention
to each step as the questionnaire or research instrument
should be adapted to the specific cultural environment
and should not be biased in terms of any one culture
(Malhotra et al., 1996).
 Questionnaire Design Process

• Designing of the questionnaire is a systematic process.

This section explores the systematic process of
questionnaire design in three phases:

• pre-construction phase,
• construction phase, and
• post-construction phase.
FIGURE 4.1 : Steps in questionnaire design process
 Phase I: Pre-Construction Phase

• Phase I is the pre-construction phase of the

questionnaire design process. It consists of three steps:

• Specific required information in the light of research

objective,

• an overview of respondent’s characteristics, and

• decision regarding selecting an appropriate survey

technique.
 Step 1: Specific Required Information in the Light
of Research Objective
• The first and the foremost important decision relates to
finding the required information through a
questionnaire.
• To generate specific information, the researcher should
clearly define the objective and the other research
components such as theoretical model, research
questions, and hypotheses.
• Clarity in all these components is very important as
these only will provide the base of specific information
to be collected through survey.
 Step 2: An Overview of Respondent’s Characteristics

• While collecting the information, an overview of the

respondent’s characteristics is a vital consideration.
• A researcher must construct the questionnaire in the
light of the respondent’s profile.
• Even while constructing a questionnaire, a researcher
has to keep the profile of a probable respondent in mind.
• Various factors such as the respondent’s qualification,
age, experience, income, marital status, occupation,
and so on have a decisive and impactful role in
conducting a research.
 Step 3: Decision Regarding Selecting an Appropriate
Survey Technique

• While constructing a questionnaire, a deep thinking

process is required to select an appropriate survey
technique.

• The questions in the questionnaire must be

constructed, sequenced, and placed according to the
mode of the survey.
 Phase II: Construction Phase

• Phase II is the real construction phase of the questionnaire

design process.
• It consists of six steps:
• Decision regarding question format :structured questions versus
unstructured questions
• Decision regarding question relevance and wording
• Decision regarding question sequencing
• Decision regarding question response choice
• Decision regarding the questionnaire layout
• Producing first draft of the questionnaire.
 Step 1: Decision regarding question format: structured questions
versus unstructured questions

• Questionnaires use two types of question formats. These are open-

ended questions and closed-ended questions.
• In the case of a “closed-ended” question, the respondent has to
format the judgment to fit the response categories and when
“open-ended” questions are used, the judgment has to be
verbalized into a preliminary answer (DeLeeuw, 2001).
• The closed ended question format can be further divided into
dichotomous, multiple-choice questions, and scales. The following
sections focus on open-ended questions and closed-ended
questions.
 Open-ended Questions

One of the major limitations is to handle the interviewer and

the interpretation bias.
 Closed-ended Questions
• Closed-ended questions are structured questions. The question
structure does matter (Connolly et al., 2005).
• The closed-ended questions provide response alternative to the
respondents instead of giving them a free-to-express response
option.
• The choice offered to the respondents can be either in the form of
a rating system or a set of response alternatives.
• These response alternatives are presented to the respondents and
they select the most appropriate one.
• The closed-ended questionnaires are generally cheaper, more
reliable, and faster to code, and analyse the collected data (Swamy,
2007).
• The closed-ended question format can be further divided into
dichotomous, multiple-choice questions, and scales.
 Dichotomous Questions

Dichotomous questions have only two response alternatives usually

presenting the two extremes “yes” or “no.” To make the alternatives
balanced, the researchers often present a third neutral alternative
“don’t know.”
Multiple-Choice Questions
 Step 2: Decision Regarding Question Wording
• The negative wordings impact the process of interpreting
the questions, leading at least some respondents to
misinterpret how to respond and, thus, reducing or
destroying the usefulness of the questions or the series of
questions (Johnson et al., 2004).
• In some cases of conducting personal interviews, the
researchers feel that they will be able to overcome the
wrong wording during the discussion, but they suggest a
different understanding.
• The effects of question wording on response accuracy did
not seem to be moderated by the respondent’s knowledge
gained via discussion (Bickart et al., 2006).
Decision parameters regarding question wording
Question Wordings Must Be Simple and Easy to Understand
 Vague or Ambiguous Words Must Be Avoided

Some words such as “often,” “occasionally” and “usually,”

“how long,” “how much,” and “reasonably well” may be
confusing for a respondent because these words specify a
specific time frame.
 Double-Barrelled Questions Must Be Avoided

Double-barrelled questions are those with wordings such as “and” or “or.”

In a doublebarrelled question, a respondent may agree to one part of the
question but not to the other part.

√
 Avoid Leading and Loaded Questions
A leading question is the one which clearly reveals the researcher’s
opinion about the answer to the question.

Identifying the loaded question bias in a question requires more

judgment because the wording elements in a loaded question allude to
the universal belief or rules of behaviour (Burns & Bush, 1999).
 Split-ballot Technique

• A leading question generally emphasizes either the positive or

negative aspect of the question.
• Sometimes, it becomes necessary to ask a question with either
the positive or negative aspect.
• In this situation, a split-ballot technique is used to avoid bias due
to positive or negative aspect of the question.
• This technique involves the construction of a single question in
two alternative phrases, and the question based on one phrase is
administered to half of the respondents and the question based
on the other phrase is administered to the other half of the
respondents.
Split-ballot Technique
 Avoid Using Overstated Words

The answer will always be overblown due to the first part of the
question, which generates a worry in the mind of the respondent and
results in a positive answer, which is not possible otherwise. A more
poised way of asking the same question is shown below.

√
 Implied Assumptions Must Be Avoided

Above question has an implicit assumption that the discount policy on

bulk purchase offered by Company “A” is working excellent and by
answering “yes,” the company will continue its policy.

√
 Respondent’s Memory Should Not Be Overtaxed

√
 Generalization and Estimation Must Be Avoided

Generalization means respondent’s belief, “what must

happen” or “what should happen.”

√
 Respondent’s Ability to Answer Must Be Considered
A question targeted to officers older than 55 years to assess
the importance of Internet banking is as follows:

×
Targeting following question to young respondents may not be an
appropriate choice.

×
 Step 3:Decision Regarding Question Sequencing

• Question sequence also plays a key role in generating the respondent’s

interest and motivation to answer the question. Questions should have
a logical sequencing in the questionnaire and should not be placed
abruptly.
• To facilitate the responses, a researcher has to follow some logical steps
in sequencing the questions in the questionnaire. This arrangement
usually requires the considerations as shown in Figure 4.3.
FIGURE 4.3:Decision parameters regarding question sequence

Decision regarding question sequencing

Difficult to Identification
Screening Opening Transition and Logical order
answer categorization of questioning
questions questions statements
questions questions
 Screening Questions
• Researchers generally begin with some screening questions
to make sure that the target respondent is qualified for the
interview.
• In some cases, when the researcher is very sure about the
qualification status of the respondent, he or she does not
incorporate the screening question and starts from some
“opening questions.”
 Opening Questions
 The opening questions should be simple, encouraging, and trust
building. From the research objective point of view, these questions
may sometimes be little irrelevant but should be good initiators.
 These questions should not seek in-depth information and should
be as general as possible.
 For example, a microwave company, trying to assess “shift in
consumer attitude” from traditional way of cooking, should ask a
first opening question as follows:
 Transition Statements
• The movement from one set of questions to another set requires transition
statements.
• For example, a mineral water bottle manufacturing company is encouraged
with the expanding market.
• The company wants to assess the potential future market and hence conducted
a survey on non-users.
• Its researchers have identified various variables to get the potential use, of which
“awareness” and “taste” are important.
• It has prepared the first 11 questions with the first 5 questions based on the
“awareness” and the next 6 questions on “taste.”
• After asking the first set of 5 questions, a researcher moves to the second set of
6 questions to get the potential consumer feeling for mineral water taste.
• Thus, before asking the next set of 6 questions, a transition statement is
required to make the respondent familiar with the coming questions.
• Hence, a transition statement “Now, I would like to understand your opinion
about the mineral water taste” will develop respondent’s connectivity for the
next set of 6 questions related to “taste,” and he or she will be in a comfortable
state of mind to answer these questions.
 Difficult to Answer Questions
• Difficult to answer, sensitive, or complicated questions should
be placed later in the questionnaire.
• Placing it first will confuse the respondent and he or she will tend
to terminate the interview process.
• Asking difficult questions first in a telephone interview reduces a
respondent’s interest in the interview process and he or she
tends to terminate the interview. Under telephone interview
conditions, substantively related questions affect the responses
to the target question only when asked first (Schwarz & Hippler,
1995).
 Identification and Categorization Questions

• Identification questions are used to generate some basic

identification information such as name, mailing address, office
phone number, personal phone number, or cell phone number.
• Categorization questions are mainly used to generate
demographic information.
• For example, researchers generally want to generate the
information related to age, experience, gender, and occupation
of the respondents.
 Logical Order of Questioning
• In a questionnaire, the questions must flow in a logical
sequence. There are at least three approaches to suggest
the roadmap to place the questions in a logical sequence;
they are funnel technique, work technique, and sections
technique.
• Funnel technique suggests asking general questions first
and then the specific questions.
• Work technique suggests that difficult-to-answer,
sensitive, or complicated questions should be placed
later in the questionnaire.
• The third technique is the section technique in which
questions are placed in different sections with respect to
some common base.
 Step 4: Decision Regarding Question Response Choice

• It is important to understand that too many

response choices will burden the respondent and
he or she will be perplexed while answering.
• Few response choices will not be able to cover all
ranges of possible alternatives.
• As a general rule, the researchers present a
question with five to seven response alternatives.
 Step 5: Decision Regarding Questionnaire Layout

• Questionnaire layout is important to enhance the response rate. A

recent study (Lagarace & Washburn, 1995) revealed that a user-
friendly format, and to some extent colour, is valuable to increase
mail survey response rate.
• The appearance of a questionnaire is particularly important in mail
surveys because the instrument, along with the preliminary letter
and/ or cover letter, must sell itself and convince the recipient to
complete and return it (Boser, 1990).
• It has been observed that the respondent emphasizes the questions
that are placed at the top of the questionnaire compared with that
at the bottom.
 Step 6: Producing First Draft of the Questionnaire

• Printing on a poor, quality paper or an unprofessional look of

the questionnaire may generate a non-serious feeling among the
respondents.
• So, the questionnaire may be printed on a good, quality paper
and must have a professional look.
• The appearance of the front cover on a mail questionnaire and
the nature of first questions have been purported to have an
important influence on the respondent’s decision to complete
the questionnaire (Frey, 1991).
 Phase III: Post-Construction Phase

• Phase III is the post-construction phase of the questionnaire design

process. It consists of four steps:
1. Pre-testing of the questionnaire
2. Revisiting the questionnaire based on the inputs obtained from
the pre-testing
3. Revising final draft of the questionnaire
4. Administering the questionnaire and obtaining responses.
 Step 1: Pre-Testing of the Questionnaire

• Pre-testing of the questionnaire involves administering

the questionnaire to a small sample of the population to
identify and eliminate the potential problems of the
questionnaire, if any.

• Testing a questionnaire can be a time-consuming process,

and this stage of questionnaire has often been
overlooked in the past in researcher’s eagerness to start
collecting data (Williams, 2003).
• Researchers generally use two common procedures to
pre-test: protocol analysis and debriefing.
 Step 1: Pre-Testing of the Questionnaire (Cont.)
• Using protocol analysis, a researcher asks the respondent to “think
aloud” while answering the question.
• Debriefing is an interview conducted when a respondent has
filled the questionnaire.
• After completing, the respondents are informed that the
questionnaire they have just filled was a pre-test questionnaire
and are requested to share their views about various dimensions
of the questionnaire.
• They are also requested to find out the problems with the
questionnaire, which they realized while filling it.
• While taking a decision about the sample size for pre-testing, as a
thumb rule, a researcher should keep the nature of population
diverse in mind.
• More heterogeneity in population requires relatively large size of
the sample compared with the situation when population is
relatively homogeneous.
 Step 2: Revisiting the Questionnaire Based on the Inputs
Obtained from Pre-Testing
• To enhance the accuracy, after incorporating suggestions
from the pre-testing, a researcher can go for second pre-
testing.
• It is always possible that the second pre-testing may also
reveal some of the problems of the questionnaire.
• Pre-testing of the questionnaire might have provided
many inputs in a subjective manner.
• At this stage, the researcher must objectively
incorporate all the inputs obtained from the pre-testing
exercise.
• All the parameters related to the question wording (as
discussed in the previous sections) must be carefully
considered.
 Step 3: Revised Final Draft of the Questionnaire

• At this stage, the researcher once again carefully

examines the questionnaire.
• Insertions and deletions of the previous stage must be
re-checked to provide the desired accuracy.
• The researcher makes the questionnaire “ready to
administer” by eliminating all the minute mistakes and
tries to make it error free.
• After careful examination of all the incorporations
obtained from pre-testing, the researcher is now ready to
have the final draft of the questionnaire and administers
to the sample taken from a target population.
 Step 4: Administration of the Questionnaire and
Obtaining Responses

• At this stage, the researchers administer the

questionnaire to the respondents and obtain the
responses.
• These responses are coded, data are tabulated,
and appropriate statistical techniques are applied
to analyse the data.