Questionnaires: Design

and Administration
MDSC 3200
Douladel P. Tyndale (Mrs.)
douladel.willie02@uwimona.edu.jm

By the end of this session and with supplementary reading, students

should be able to:
Explain what a questionnaire is
Outline the steps involved in questionnaire construction
Differentiate between: open and closed ended questions; likert-type
and forced- choice response formats
Discuss the art of proper sentence construction and identify the
type of errors in wording of questions
Differentiate between the 2 main modes of questionnaire
administration
Discuss issues of bias in questionnaire design and administration
and how it may be reduced

Learning Objectives

 Perhaps epidemiologys most fundamental role is to

provide a logic and structure for the analysis of health
problems both great and smallWe have to think about
what is worth counting and how best to count it
- Pirozzo et al, 2005 Essential epidemiology

Introduction

Techniques for data collection

Questionnaire administration
Interview
Observation
Direct physical/chemical measurement
Record review
Etc.

Introduction

 A data collection instrument, in the form of a

document, with questions and statements to which
people provide written responses in their own
words or by selecting from predefined answers

What is a Questionnaire?

Four main steps

1. Specifying of information being sought
2. Initial drafting
3. Pre-testing
4. Redrafting and finalization

Questionnaire Construction

1. Specifying of information
Guided by research objectives
Thinking and discussion with experts
Literature review

Process culminates with list of information to be later

translated into specific questions

Steps in questionnaire
construction

2. Initial drafting
List from previous step carefully crafted in questions
Deliberate decisions re
Words and phrasing
Order of questions
Degree of complexity
(implications for validity - see later slides)

Steps in questionnaire
construction

3. Pre-test

Administer questionnaire to

Research colleagues and subject matter experts

Small group of persons similar to group that will be studied

Aimed at

Improving clarity
Detecting errors and other issues
Receiving feedback on questions

Important for newly developed instrument or firsttime use in particular population/setting

Steps in questionnaire
construction

4. Redrafting and finalization

Adjustments made based on results of pre-test

exercise

If major, revise and repeat pre-test

If minor, revised questionnaire can be

administered to full sample

Steps in questionnaire
construction

It is worth noting that

although questionnaires
may appear to be
deceptively easy to
construct, a well
designed instrument
requires considerable
effort

Constructing a
questionnaire and
wording the
questions are not
simple tasks
Meadows, 2003

Coates, 2004

Words of wisdom

 Open ended
How do you feel about the standard of the treatment you
received while you were a patient at this hospital? *

Closed ended
How would you rate the standard of the treatment you
received while you were a patient at this hospital? (circle
one number) 1-excellent; 2-good; 3-moderately good; 4fair; 5-poor*

Types of Questions
*Introduction to Research in the Health
Sciences, 5th ed. Polgar & Thomas

Open ended

Closed ended

No pre-determined
responses
Less/un-structured
Benefits

Pre-determined list of response

options
Benefits

More detailed responses elicited

Costs
Responses difficult to encode
and analyse with quantitatively
Greater respondent burden
(longer time to answer,
difficulty in gathering and
writing thoughts)

Tightly structured
Responses easily encoded and
analysed
Less time and effort to complete

Costs
Less in-depth responses
Researcher must ensure
completeness of options
Respondents must fit their answer
in provided responses

Types of questions

Can frustrate respondent and/or yield

less than accurate results

Likert

Forced-choice

Conventional 5-point
Likert-type scale
*My medical practitioner
always explains the chosen
treatment to me (circle one
number).

Four-point forced choice

scale
*My medical practitioner
always explains the chosen
treatment to me (circle one
number).

1 Strongly agree
2 Agree
3 Undecided
4 Disagree
5 Strongly disagree

1 Strongly agree
2 Agree
3 Disagree
4 Strongly disagree

Response formats - attitudinal questions

*Introduction to Research in the Health
Sciences, 5th ed. Polgar & Thomas

Likert-type
Advantage
Allows undecided
responses

Disadvantage
Does not guard against
acquiescent response mode
(phenomenon where
respondents give middle
answers all the time)

Forced-choice
Advantage
Respondent forced to give a
positive or negative
response

Disadvantage
Undecided response not
allowed

Likert and forced-choice response

formats
Note: Another observed phenomenon is extreme response mode where a respondent
never selects an intermediate point on the rating scale

 Double-barrelled questions
Two questions included in one: Do you like epidemiology or
biostatistics?
What is the problem here? How should a yes be interpreted?

Ambiguous questions
Questions having terms that are likely to mean different
things to different people because of different reference
points: Do old people?

Wording of Questions
-Pitfalls to avoid

 Complicated questions (Level of wording)

Avoid jargon and (uncommon) abbreviations: Do you think DNA
typing should be mandated for all persons convicted of a crime?
Consider your audience, decide wisely

Leading questions
Questions that appear to lead the respondent to answer in a
particular way: How often do you? This can be made worse if
never is not a response option
Pattern of questions can create bias is persons are led to respond
in a particular way: Do you value human life? Do you think
unborn babies should be murdered?
This question is also too emotive

Pitfalls to avoid

 Unnecessarily wordy questions

Over the past year have you attended any hospital including
your local hospital, but not including accident and
emergency departments, to visit friends or family?

Negatively worded questions

I should not take my medication if I am feeling fine
Strongly agree/agree/Undecided/disagree/strongly disagree

Instead, try:
I should take my medication if I am feeling fine
Strongly agree/agree/Undecided/disagree/strongly disagree

Pitfalls to avoid

 Example of a commonly used format:

Introductory statement: purpose, general info being sought
and by whom, confidentiality statement
Demographic questions: age, sex, education, etc
Factual questions: easy to answer, positioned early as warm
up
Opinion questions: may require reflection, respondent
perhaps more comfortable now
Sensitive questions: usually appear at a point where
respondent expected to be somewhat comfortable. Preamble usually used
Closing statements and return instructions

Structure of questionnaire

 Divide into sections with common focus with appropriate

lead-in and instruction for each
Skip patterns to lead respondents to appropriate questions
based on previous responses. Use with caution: trade off
between reducing respondent burden and causing
confusion
5. Do you smoke cigarettes? Yes
No
If, No to 5, please go to question 8
(What types of questions do you think are in 6 and 7?)

Useful techniques

 Never undermine the power of aesthetics (selfadministered instruments: appropriate spacing, font and
font-size, no unnecessary clutter, general appearance)
Keep questionnaire as short as possible (Can affect the
response rate!)
Mode of administration (next slide) usually dictates how
a questionnaire is organised and presented

Other tips

Self-administered

Interviewer-administered

Respondent completes on own

Cheaper to administer
Not susceptible to interviewer
bias
Ideal for sensitive issues
Respondents must be literate
No/little opportunity to seek
clarification
Higher refusal rate (especially if
mailed)
More prone to errors in
completing

Interviewer asks questions

More expensive to
administer
Prone to interviewer bias
Not ideal for sensitive issues
More opportunities to seek
clarification
Lower refusal rate
Greater chance of being
correctly filled out

Administering a questionnaire

 Questionnaires rely on honesty and accuracy of recall from

the respondent. Some information points can be verified,
others cannot or will not
Information bias:
Recall bias
Respondents might be asked about past experiences. There may
be differences in the ability to accurately recall among
respondents and experiences

Interviewer bias
Interviewer can consciously or unconsciously influence
responses away from the truth
Possibly because of tone, disposition, body language, or mere
presence

Issues with bias and validity

 Social desirability bias

Respondents report inaccurate information based on what they
think is socially acceptable
Most commonly seen as over reporting of good behaviour and
under reporting of bad behaviour or variation in reporting of same
type of behaviour by different groups
Eg. In studies of sexual practices - # of sexual partners in the last
12 months. (women tend to under report, men tend to over
report)

Selection bias
Who are the persons who complete the questionnaire? How
are they different from the ones who do not?

Issues with bias and validity

 Proper sentence and questionnaire construction

Appropriate consultation with literature and experts
Pre-testing and necessary revision

Clear and complete instruction for respondents and (if used)

interviewers
Thorough training of interviewers (if used)
Objectivity
Standardized way of asking questions, giving prompts

Post-hoc data analyses can check for interviewer bias for

example compare responses between interviewers (Cant undo
if present but can make decisions about accepting study data)

Dealing with bias and issues of validity

and quality

Choose from A-E for questions 1 and 2 below.

A. Double-barrelled
B. Leading
C. Too technical
D. Ambiguous
E. Biased
1. Do you attend gay parties?
2. Are you presently taking -blockers?

Test Yourself

 We didnt specifically discuss questionnaires that are

designed into scales which allow for a score to be
computed based on responses. The general rules of
questionnaire construction apply. These instruments
further undergo psychometric testing where their
reliability and validity are assessed. This will be
discussed in the lecture on validity and reliability and
also in Seminar 6.

Note

 Polgar and Thomas 2008. Introduction to Research in the

Health Sciences. 5th ed

References