Vous êtes sur la page 1sur 36

Developing a Questionnaire

MELJUN CORTES
MELJUN P. CORTES MELJUN P. CORTES

MELJUN P. CORTES

MELJUN P. CORTES

Types of Questions

Open-ended
high

validity, low manipulative quality

Closed-ended
low

validity, high manipulative quality

MELJUN P. CORTES

MELJUN P. CORTES

Open-ended

An open-ended question is one in which you do not provide any standard answers to choose from.
1. How

old are you? ______ years. 2. What do you like best about your job?

MELJUN P. CORTES

MELJUN P. CORTES

MELJUN P. CORTES

MELJUN P. CORTES

MELJUN P. CORTES

MELJUN P. CORTES

Closed-ended

A closed-ended question is one in which you provide the response categories, and the respondent just chooses one:
What do you like best about your job? (a) The people (b) The diversity of skills you need to do it (c) The pay and/or benefits (d) Other: ______________________________

MELJUN P. CORTES

MELJUN P. CORTES

MELJUN P. CORTES

MELJUN P. CORTES

MELJUN P. CORTES

MELJUN P. CORTES

MELJUN P. CORTES

MELJUN P. CORTES

Dichotomous Questions

Dichotomous Question: a question that has two possible responses only


Could

be Yes/No True/False Agree/Disagree


MELJUN P. CORTES

MELJUN P. CORTES

Interval Level

Attempt to measure on an interval level Likert response scale: ask an opinion question on a 1-to-5, 1-to-7, etc. bipolar scale Bipolar: has a neutral point and scale ends are at opposite positions of the opinion
Semantic

differential: an object is assessed by the respondent on a set of bipolar adjective pairs scale: respondent checks each item with which they agree; constructed as cumulative, so if you agree to one, you probably agree to all of the ones above it in the list
MELJUN P. CORTES

Guttman

MELJUN P. CORTES

MELJUN P. CORTES

MELJUN P. CORTES

RESPONSE OPTIONS:
Strongly Disagree 1 2 Neutral 3 4 Strongly Agree 5

New FourPoint Likert Scale


Strongly Disagree Disagree Agree Strongly Agree

MELJUN P. CORTES

MELJUN P. CORTES

.14

MELJUN P. CORTES

MELJUN P. CORTES

Questions based on Level of Measurement

Use a nominal question to measure a variable Assign a number next to each response that has no meaning; simply a placeholder. Use an ordinal question to measure a variable Rank order preferences More than 5 10 items is difficult Does not measure intensity
MELJUN P. CORTES

MELJUN P. CORTES

Filter/Contingency Questions

To determine if a respondent is qualified to answer questions, might need a filter or contingency question (also known as knowledge)
Limit

# of jumps

If
If

only two levels, use graphic to jump

you can't fit the response to a filter on a single page, it's probably best to be send them to a page, rather than a question #
MELJUN P. CORTES

MELJUN P. CORTES

How many steps in the response scale?

Statistical reliability of the data increases sharply with the number of scale steps up to about 7 steps
After

7, it increases slowly, leveling off around 20, it decreases sharply

11
After

MELJUN P. CORTES

MELJUN P. CORTES

Should there be a middle category?


Does it make sense to offer it? Should not be used as the dont know or no opinion option.
The

middle option is usually placed between the positive and negative responses.
its last in an interview.
MELJUN P. CORTES

Sometimes
MELJUN P. CORTES

Direct Magnitude Scaling

Method of obtaining ratio-scaled data


Idea

is to give respondents an anchor point, and then ask them to answer questions relative to that

Example:
Suppose

you are interested in the severity of crimes. Begin by assigning a number to one crime and then have respondents assign numbers to the others based upon a ratio.
MELJUN P. CORTES

MELJUN P. CORTES

Filtering "Don't Know"

Standard format
No

"don't know" option is presented to the respondent, but is recorded if the respondent volunteers it.

Quasi filter
A

"don't know" option is included among the possible responses.

Full filter
First

the respondent is asked if they have an opinion. If yes, the question is asked.
MELJUN P. CORTES

MELJUN P. CORTES

Question Placement

It's a good idea to put difficult, embarrassing or threatening questions towards the end
More If

likely to answer.

they get mad and quit, at least you've gotten most of your questions asked!

Put related questions together to avoid giving the impression of lack of meticulousness
Watch out for questions that influence the answers to other questions.
MELJUN P. CORTES

MELJUN P. CORTES

Wording of Questions

Direction of Statements
Response

bias Socially desirable

Always and never


Avoid

this Better to phrase as most, infrequently

Language
Reflect

educational level and reading ability Need for various languages


MELJUN P. CORTES

MELJUN P. CORTES

Frequency and Quantity

Consider both frequency and quantity


Consider
Consider

number of times
duration of times

MELJUN P. CORTES

MELJUN P. CORTES

Mutually Exclusive and Exhaustive

Mutually exclusive: not possible to select more than one category/value Exhaustive: providing all possible categories/values

MELJUN P. CORTES

MELJUN P. CORTES

Forced Choice

Choose between 2 choices


Might
Other

not be relevant
choices exist (or at least possible)

Lesser

of two evils

MELJUN P. CORTES

MELJUN P. CORTES

Recalling Behavior

Can be difficult to remember

Ask questions that can be answered


Choose time frames that are reasonable Pilot test for time frame issues

MELJUN P. CORTES

MELJUN P. CORTES

Response Bias

Exaggerating the truth

Socially desirable answers


Consider using trap questions
Possibly

fictional choice

MELJUN P. CORTES

MELJUN P. CORTES

Sensitive Items

More comfortable answering in categories


Minimize
Might

missing data

loose statistical power

MELJUN P. CORTES

MELJUN P. CORTES

Evaluating Questions
Pre-testing Cognitive interviewing Behavior coding Peer review

Peer review has shown to be the best method but its the least used.
MELJUN P. CORTES

MELJUN P. CORTES

Validity and Reliability of Questions

Evaluative strategies:
Analysis

of data to evaluate the strength of predictable relationships among answers and with other characteristics of respondents. of data from alternatively worded questions asked of comparable samples. of answers against records.

Comparisons

Comparison Measuring

the consistency of answers of the same respondents at two points in time.


MELJUN P. CORTES

MELJUN P. CORTES

Coding the Questionnaire

Create a codebook: reference guide for the data set Code: assigning a value to a response category
Often

numeric code Pre-coding makes it easier Content analysis on open-ended items Yes/No often coded as present or not (0 or 1)
MELJUN P. CORTES

MELJUN P. CORTES

Missing Responses

Why blank?
Missed

them Refusal to answer Didnt feel it applied Didnt know the answer

To code or not
Analyze

the difference If know why, might consider


MELJUN P. CORTES

MELJUN P. CORTES

Piloting the Questionnaire

Test it on yourself
Possibly

other experts

Test on people similar to sample


Dont

reuse (some exceptions)

Discuss the survey with individuals


During

completion or After
MELJUN P. CORTES

MELJUN P. CORTES

Finding Respondents

Best Methods of Selection Even with a good survey, poorly chosen sample leads to poor results

MELJUN P. CORTES

MELJUN P. CORTES

MELJUN P. CORTES

MELJUN P. CORTES