Survey and Research Methods

Session 5: Questionnaire Design

Dr. Gangadhar Mahesh, B. Arch, M. Eng, Ph.D

NICMAR
ACM Survey and Research Methods

1 Dr. Gangadhar Mahesh

Questionnaire
Is a group or sequence of questions designed to obtain information on a subject from a respondent
- Include forms used for sample surveys and censuses as well as administrative forms - questions posed must conform to the surveys Statement of Objectives and provide information that is useful for data analysis - All information requirements must be met, but each question should have an explicit rationale for being on the questionnaire - questions must be sequenced logically for the respondent - questions must be designed so that they are easily understood and can be accurately answered by respondents - should be tested before implementation using cognitive testing, focus groups, informal testing

A well-designed questionnaire should:

- collect data efficiently with a minimum number of errors and inconsistencies - be respondent friendly and interviewer friendly (if interviewer-assisted) - lead to an overall reduction in the cost and time associated with data collection

NICMAR
ACM Survey and Research Methods

2 Dr. Gangadhar Mahesh

Questionnaire Design 1
Questionnaire design process starts with the formulation of survey objectives and information requirements and continues with the following steps:
1. Consult with data users and respondents 2. Review previous questionnaires
- other surveys on the same or a similar topic can be a useful starting point when considering how to draft a question - when trying to compare results from different surveys, the same questions should be used - documentation on the data quality from such surveys should be examined to assess the effectiveness of the questionnaire

NICMAR
ACM Survey and Research Methods

3 Dr. Gangadhar Mahesh

Questionnaire Design 2
3. Draft the questionnaire
- Method of data collection - the availability of the data being sought - Characteristics of respondents - Response burden - Complexity of the data to be collected - Confidentiality and sensitivity of the information - Translation - Comparability of results with other surveys - Consistency - Respondents willingness to provide the data - The likelihood of nonresponse - Administrative requirements - The type of questions, the wording of each question - The layout of the questionnaire - Sources of measurement or response error - Processing of the questionnaire.

NICMAR
ACM Survey and Research Methods

4 Dr. Gangadhar Mahesh

Questionnaire Design 3
4. Review and revise questionnaire
- should identify any obvious problems with the questionnaire such as mistakes in spelling or grammar or awkward wording - could include experts in the field of study, people who have experience designing questionnaires, interviewers or members of the population to be surveyed

5. Test and revise questionnaire

- on typical respondents and on specific subpopulations that may have problems with particular questions - intended to identify difficulties and errors with the questions - identify whether the question order affects the interpretation of questions, whether instructions are clear and how respondents feel about the appearance of the questionnaire.

6. Finalise questionnaire

NICMAR
ACM Survey and Research Methods

5 Dr. Gangadhar Mahesh

Questionnaire Testing Methods

Informal testing (pretesting) Cognitive methods Focus groups Interviewer debriefings

Behaviour coding of interviewer/respondent interactions

Split sample tests Pilot tests

NICMAR
ACM Survey and Research Methods

6 Dr. Gangadhar Mahesh

Types of Questions 1
Open Ended Questions
- Those where response categories are not provided

Advantages:
- Allow the respondent the opportunity for self-expression or elaboration without being influenced by the response categories that would be provided by a closed question - Allow to obtain the respondents own natural wording - Allow to obtain exact numerical data - Allow to provide follow-up to closed questions

Disadvantages
- Are burdensome since the respondent must determine the intent of the question and formulate an answer without the aid of response categories - Recording the answer is time consuming

NICMAR
ACM Survey and Research Methods

7 Dr. Gangadhar Mahesh

Types of Questions 2
Closed Questions
- Those where response categories are listed with the question

Advantages:
- less burdensome to respondents - data collection and capture is cheaper and easier - are quicker and easier to answer - more likely to answer the question as intended - easier to analyse data that has been collected using closed questions, since the answers are more consistent and already grouped - if a question is being used for several surveys, using the same response categories facilitates comparisons across surveys

Disadvantages
- effort is sometimes required to develop the response categories - the response categories must be mutually exclusive and exhaustive - can force the respondent to choose a category that does not correspond to his or her opinion, or to express an opinion when in fact he or she has none

NICMAR
ACM Survey and Research Methods

8 Dr. Gangadhar Mahesh

Types of Questions 3
Two Choice Questions
- Used to split the respondents into two distinct groups - Also used as screening questions to prevent respondents from being asked a series of questions that do not apply to them

Multiple Choice and Checklist questions

- response categories should be mutually exclusive and exhaustive

Ranking Questions
- burdensome to rank categories - sizes of the rank intervals are unknown and unlikely to be equal - a respondent may assign two or more items the same ranking

Rating Questions
- how many categories should the rating scale have? - whether or not the rating scale should have a neutral alternative?

NICMAR
ACM Survey and Research Methods

9 Dr. Gangadhar Mahesh

Scales of Measurement and Data Analysis

NICMAR
ACM Survey and Research Methods

10 Dr. Gangadhar Mahesh

 Graphic Rating Scales Likert Type Scale

- Assume that the strength/intensity of experience is linear - Is a five (or seven) point scale

Semantic Differential Scale

- used for measuring the meaning of things and concepts - is set up using polar adjectives

Unipolar Vs. Bipolar Scales

NICMAR
ACM Survey and Research Methods

11 Dr. Gangadhar Mahesh

Guidelines for Formulating Questions

Keep it Simple Define Acronyms and Abbreviations Ensure that Questions are Applicable Be Specific Avoid Double-Barrelled Questions Avoid Leading Questions
- Please tell me whether you agree with, disagree with, or have no opinion on the following: Tourism is beneficial to Northumberland County and therefore should be promoted.

Avoid using Double Negatives

- Would you be for or against not allowing the sale of alcohol in convenience stores? - Would you be for or against allowing the sale of alcohol in convenience stores?

Soften the Impact of Sensitive Questions Ensure that Questions Read Well
NICMAR
ACM Survey and Research Methods
12 12 Dr. Gangadhar Mahesh

Questionnaire Layout
Introduction Opening questions Placement of sensitive questions Placement of demographic and classification questions

Transitional Statements
Instructions Format Considerations

NICMAR
ACM Survey and Research Methods

13

13 Dr. Gangadhar Mahesh