Instrumentation PDF

INSTRUMENTATION

INSTRUMENTATION
Development and valida8on of data-‐
collec8on tools
•  Ques8onnaires
•  Interview schedules
•  FGD guides
•  Lab form sheets
•  Observa8on checklists
QUESTIONNAIRE
DESIGNS!
DEFINITION
•  A ques%onnaire is a data-‐collec6on tool used
to gather informa8on about an individual,
group, things, phenomena or events.
•  It is a wri;en document which…
1)  respondent fills in by himself (self-‐
administered) or with assistance from the
researcher (interviewer-‐assisted)
2)  collected by field interviewers (interviewer-‐
administered)
3) may be sent through standard mail or email.
Developing a ques6onnaire is not
easy. It is a tedious and 6me-‐
consuming process.
•  Use exis8ng relevant ques8onnaires

•  Modify exis8ng ques8onnaires to suit the
research requirements.
•  Items from several ques8onnaires can be
combined.
Two common mistakes made by
inexperienced inves8gators:
– asking too many ques8ons, the result of

which may never be analyzed;
– asking too few ques8ons that may be
inadequate in sa8sfying the research
objec8ves.
1. An effec8ve ques8onnaire depends on the
clarity of the research problem.

•  Make sure that you have a clear, solvable
research problem.
•  Ques8onnaire data must be able to provide

an answer to the research problem.
2. The path to solve the research problem
is through a clear statement of the
research objec8ves
– The specific research objec8ves must

be able to indicate what informa8on is
needed to solve the research problem

– Only informa8on that are relevant to
the research objec8ves should be
obtained.
EXAMPLE
Research problem: Why do persons with leprosy
delay their medical consulta8on?
Specific objec8ves:
1. To know the prevalence of people with leprosy
in the sample community.
2. To know the ra8o of pa8ents seeking medical
help vis-‐a-‐vis those who have not sought any
medical a]en8on
3. To determine their socio-‐demographic
characteris8cs.
4. To assess their percep8ons and a_tudes about
their health condi8ons
5. To iden8fy the reasons why pa8ents seek or do
not seek medical help.
The research objec6ves will help decide what
data need to be collected
HEALTH
SEEKING
Factors
in health
Prevalen Socio-‐ Percep6ons/ seeking
ce demographics aKtudes Socio-‐
cultural,
Ra6o psychological
Ra6o health-‐ Feeling
Concept . Economic,
with/ seeker/ age, sex, s about health
ses, cs, of having
without no service,
leprosy health educ disease the clinical
seeker disease
ques6ons ques6ons ques6ons ques6ons ques6ons ques6ons

3. What type of ques6onnaire do you
need?
1.  Descrip8ve ques8onnaire

2. Measurement scales

3. Combina8on of descrip8ve and scales
4. What type of ques8ons would you like to
include?
•  Open-‐ended: What is your opinion about the
present state of skin care in the Philippines?

•  Closed-‐ended: Are you in favor or not in favor
of the current skin care system?
[ ] Yes, I am in favor
[ ] No, I am not in favor
[ ] Could not say/don’t know

Par.ally closed-‐ended: Which of the following
do you think is most responsible for the
popularity of cosme8c dermatology in the
country?

[ ] endorsement from celebri8es
[ ] tes8monies of those who have undergone
cosme8c dermatology
[ ] increasing concerns of people about good
physical appearance
[ ] influx of new technologies
[ ] others _________ (please specify)
5. How would you style the ques6ons?
•  List -‐ The respondent is given a list of choices, any of
which may be selected
•  Category-‐ Only one response can be selected

•  Rank -‐Respondent is asked to place something in
order
•  Ra6ng scales -‐A ra8ng device is used to record

response
•  Quan6ty-‐ Response is a number giving the amount
List ques6ons-‐ The respondent is given a list of
choices, any of which may be selected

Please choose the types of skin diseases that are
common in this community. Use locally understood
terms
q Galis
q Leprosy o ketong
q An-‐an
q Eczema
q Acne o tagihawat
q Buni
q Psoriasis
q Others (please specify______)

Category ques6on

Each respondent’s answer can only fit one
category

How oken do you visit your dermatologist?
q  Once a month
q  2-‐3 8mes a month
q  At least once every 3 months
q  At least once every 6 months
q  Others ____

Quan6ty ques6on

•  When did you start seeing a
dermatologist? ________
•  How many 8mes a month do you go to a

skin care clinic? _____
Ranking ques6on-‐ respondent is asked to
place something in order
•  Rank-‐order the three most important reason
why you specialized in dermatology (where 1
indicates the most important reason):
q Interes8ng
q Profitable
q More job opportuni8es
q Most in demand
q Less academically demanding
q Others __

Ranking ques6on
•  Should not ask respondent to rank more

than 15 items

•  In telephone ques8onnaire, should not
ask respondent to rank more than four
responses

Ra6ng scales ques6on-‐ a ra8ng device is
used to record the response
1.  Likert type
“Sun exposure is bad for my skin”
/-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐/-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐/-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐/-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐/
Strongly Agree Neither Agree Disagree Strongly
Agree nor Disagree Disagree

My experience of depression during the past week
was:
[ ] none
[ ] a bit
[ ] moderate
[ ] a lot

2. Seman6c differen6al (SD) or adjec6val
ra6ng scale. A pair of bipolar objec8ves are
used to express the subject’s feelings about
the concept.

Our present skin care system is……….
efficient 1 2 3 4 5 not efficient
accessible 1 2 3 4 5 not accessible
of quality 1 2 3 4 5 not of quality

3. Gu;man scale – respondents rate a
variable on a sliding scale.

“How confident are you with your facial
appearance? Are you…. (Check one)
q  Unable to go out of your house at all?
q  Able to go out but with make up?
q  Able to go out even without make up?
4. Visual analogue scales

To what extent does your skin disease
affect your level of self-‐esteem?

/___________x_________________________/
Not affected extremely
at all affected
QUANTIFYING A QUALITATIVE VARIABLE
(Steps in Scale Construc8on)
1.  Determine the nature of the scale to be constructed.
2.  Opera8onally define the construct. Determine domains to
be included in the scale.
3.  Devise items for each domain.
4.  Establish content validity of the scale.
5.  Develop response categories.
6.  Translate the items, if necessary
7.  Pretest the scale.
8.  Do item analysis.
9.  Pretest scale again if necessary. Re-‐do item analysis.
10.  Establish validity and reliability of the scale.
11.  Finalize the scale.
1. Determine the nature of the scale to be
developed.
“Knowledge”
IQ test “Prac8ce”
Achievement test
Mastery Test Performance appraisal
Screening test Behavioral checklist
Compliance scale
“A_tude”
A_tude scales
Monitoring checklist
Ap8tude test Observa8on guides
Personality test

Quality of Life Scale
Interest scale

II. Opera8onally define your construct.
CONSTRUCT-‐ an a]ribute of a person, thing or event which is
not readily observable
Examples: a_tude, intelligence, quality of life, compliance,
s8gma

DOMAINS-‐ dimensions or aspects of the construct
Example: Construct-‐A_tude
Domains: Beliefs
Affect
Behavioral inten8ons

VARIABLES-‐ concrete measures or indicators of the domains

Example
Construct: Quality of life – (subjec8ve well-‐being)

Domains: Physical well-‐being
Emo8onal well-‐being
Social well-‐being
Func8onality
Cogni8ve well-‐being
Variables:
I. For emo8onal well-‐being
In the past 2 weeks,
1. did you feel sad because of your disease?
2. did you feel anxious because of your disease?
3. did you feel irritable because of your disease?

II. For physical well-‐being
In the past 2 weeks,
1. how much pain did you experience?
2. how easy were you to get 8red?
3. how oken did you vomit?
III. Devise items for each domain

1.  Construct an item pool of about 7-‐10 items per domain.
In the final scale, there should be at least 5 items lek
(aker item analysis)
2.  Items can be in the form of statement or ques8on. It
can be answerable by dichotomous responses or
mul8ple responses.
3.  Items can be derived from exploratory interviews with

experts and target respondents; review of related
literature, survey of past ques8onnaires, or own clinical
and social experiences.
4.  It should follow strictly the rules for wri8ng ques8ons

Wri8ng good ques8ons
1.  Are the words simple and direct?
2.  Is the ques8on as clear and specific as possible?
3.  Is it a double barreled ques8on?
4.  Does the ques8on have a double nega8ve?
5.  Is the ques8on too demanding?
6.  Were technical jargons used?
7.  Are the ques8ons leading or biased?
8.  Are the ques8ons applicable to all?
9.  Is the ques8on embarrassing or objec8onable?
10.  Have you assumed too much knowledge?
11.  Can the ques8on be shortened without loss of
meaning?
12.  Is an appropriate 8me referent provided?
IV. Establish the content validity of the scale
•  Make sure that the scale has covered adequately the domains
of the construct

•  Make sure that there are enough items for each domain
(about 7-‐10 items per domain)

•  Make sure that the items are relevant to the culture and
condi8on of the target respondent (emic content)
•  Seek expert opinion on the content validity of the scale
V. Develop response categories
•  The best response scale suits the level of literacy of
the target respondents.

Rule of thumb – level of language should be at grade 6
level even if target respondents are professionals

•  Psychometrically, con8nuous scales are more
preferred than discrete /dichotomous scales
TYPE OF RESPONSE SCALES
1.  Discrete, nominal scales

I feel anxious about the consequences of my disease.
[ ] True [ ] False

Do you feel ashamed of your illness?
[ ] Yes [ ] No
Hedonic Scale
2. Con8nuous scales
•  A. Visual analogue scales
How much pain have you felt in the past 2 weeks?
__________x_______________________________
No pain at all extreme pain

B. Adjec8val scales (Likert)

My social ac8vi8es are limited by my disease.

____/______________/__________/_____________/_____________/___
Strongly agree agree undecided disagree strongly disagree

C. Faces scale
Do you feel anxious about your disease?

10 9 8 7 6 5 4 3 2 1 0
D. Ladder scale
Hedonic Scale
VII Pretest the scale
•  1. to self
•  2. to colleagues
•  3. to sample of target respondents

Sample size: number of items x 3

Take note of comments about item wording,
transla8ons, format, styles, size of font, color
of paper
The following ques6ons can be addressed
through pilo6ng:

•  Do the respondents understand what the
ques8onnaire is about?
•  Do they feel comfortable answering the

ques8ons?
•  Is the wording of items clear?
•  Do the respondents draw the same meaning

as you intended?
•  Which items or sec8ons are most difficult to answer?
•  Which seem to produce irrita8on, embarrassment or
confusion?
•  Are there any items that the respondent considers
irrelevant?
•  Is the ques8onnaire too long?
•  Do the filter ques8ons and skips work properly?
•  Are the instruc8ons clear?
•  Are the transi8ons from ques8on to ques8on and
sec8on to sec8on smooth?
VIII-‐ Code your data into a data entry program
-‐May use Excel, d-‐Base, SPSS or other spreadsheets
-‐Recode items if posi8ve and nega8ve statements are used

Example:
I feel sad that I am afflicted by this disease. (nega8ve statement-‐ as stated in the
original scale)
1-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐2-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐3-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐4-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐5
Strongly agree undecided disagree strongly
agree disagree

I feel op8mis8c about my health condi8on. (posi8ve statement-‐ recoded)
5-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐4-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐3-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐2-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐1
Strongly agree undecided disagree strongly
agree disagree

IX. Do item analysis
1.  To reduce number of items

2.  To establish validity and reliability of the
scale
Selec8ng items for the final scale
Qualita8ve item analysis

-‐based on results of cogni8ve debriefing

Are there any words or items that are ambiguous?
Do the respondents infer a different meaning to the ques8ons
than intended?
Are there unfamiliar terms?
Do the ques8ons ask for more than one thing?
Is the item difficult to understand? Difficult to answer?
Are there items which are not applicable to all?
(See Guide for developing good ques8ons)

Quan6ta6ve item analysis
1.  Reliability analysis

-‐ internal consistency (homogeneity) of
items
(Do the items measure the same trait?)

-‐ temporal stability of scores
(Are the scores the same when test is administered
on 2 different occasions)

Measures of internal consistency
(homogeneity)
1. Frequency endorsement – frequency distribu8on

2. Inter-‐item correla8on-‐
Correla8on of each item with another item in the same scale

3. Item-‐total correla8on
-‐  Correla8on of each individual item and the total score minus
the score for that item.
4. Cronbach’s coefficient (for scales with mul8ple response

op8ons) or Alpha/Kuder Richardson 20 (for scales with
dichotomous response op8ons)
Internal consistency of the whole scale
Temporal stability of scores
•  Test-‐retest method
No significant difference in scores when same
test is administered in 2 separate occasions to
same subjects using the same measurement
scale
Validity
Content validity
1. Does each item pertain to one aspect of the construct
(content relevance)?
2. Are all domains covered by at least 5 items (content

coverage)?

3. Was this assessed by a panel of experts?

4. Were the items derived or pretested with a representa8ve
sample of target respondents?
Criterion-‐related validity
1. Concurrent validity-‐ How well do the results

correlate with other similar measures of the
construct (e.g., clinical ra8ngs, other
measurement scales, lab results)?
2. Predic8ve validity-‐ How well do the results

able to predict future performance?
Construct validity
1.  Do extreme groups get significantly different scores?
Example: Do treated subjects score be]er than untreated

subjects?
2.  Do the scores change with exposure to the

interven8on?
3.  Does the scale not correlate with other indices that
measure different a]ributes (divergent validity)?
4.  Are the results consistent with theory or social/clinical

expecta8ons?
Methods of reliability analysis: internal
consistency (homogeneity)
1. Frequency endorsement – item is endorsed by less than 10% or
more than 90% of respondents

2. Inter-‐item correla8on-‐
Correla8on of each item with another item in the same scale
– good correla8on of 0.40-‐0.60

3. Item-‐total correla8on
-‐  Correla8on of each individual item and the total score minus
the score for that item.
-‐  -‐good correla8on of 0.20 and above
4. Cronbach’s coefficient or Alpha/Kuder Richardson 20-‐ internal
consistency of the whole scale where alpha must be 0.60 and
above.
Frequency endorsement
Good item

Bad item
ITEM-‐TOTAL CORRELATION
Correla8on of each individual item and the total score minus the
score for that item.
-‐good correla8on of 0.20 and above

correlate A1, Atotal r=0.29

correlate A3, Atotal r=0.10 possible item for dele8on
SUMMARY of RELIABILITY ANALYSIS
DOMAINS (N=39) ITEM-TOTAL ALPHA

CORRELATION
A. Social factors 0.83-0.89 0.9571
B. Affect 0.52-0.87 0.9003

C. Personal beliefs 0.42-0.74 0.8088
D. Community beliefs 0.61-0.87 0.8607
E. Personality (n=6) 0.65-0.84 0.8514

F. Environmental 0.78-0.89 0.9503
Influences
Procedures for reliability analysis: Interpreta8on
1.  If item is “bad” in all methods, delete item
2.  If item is “bad” in frequency endorsement and inter-‐

item correla8on, reconsider using results of item-‐total
and Cronbach’s alpha.
3.  If item is “bad” in item-‐total and Cronbach’s alpha,

delete item or modify depending on magnitude of
“badness” (e.g., very low correla8on)
4.  If item is “bad” in several or all methods but is
considered cri8cal in terms of clinical or social
significance, then consider the item but analysis will be
descrip8ve for that par8cular item (i.e., do not include
the item in the computa8on of the overall numerical
scale score).
Issues related to scoring
1.  If items are internally consistent, then a

composite single score (index score) can
be generated
-‐per domain
-‐ for the whole scale
can be expressed as
-‐ summa8ve score
-‐ mean or median scores

Issues on scoring
2. To interpret, index score can be treated as a

con8nuous variable
OR
if scores need be classified as low, moderate
and high,
-‐ use interval scales if wish to establish a gold
standard
-‐ use standard devia8on if scores are
normally distributed
-‐ use percen8les if distribu8on is skewed.
Issues on scoring
If items are not internally consistent (i.e., low alpha,
low item-‐total correla8on), then use descrip8ve
method of scoring and interpre8ng results

To do this:
-‐ analyze each item by ge_ng the percentage of
endorsement of each response op8on

-‐ or analyze each item using mean or median item
scores

-‐ do content analysis
8. Ques8onnaire must have a good format

POOR FORMAT: "What is your approximate monthly
income?"
____ Under P 5,000 ____ P 5,000 -‐ P 9,999
____ P10,000 -‐ P 19,999 ____ P20,000 or more

BETTER FORMAT: "What is your approximate monthly
income?”
[ 1 ] Less than P 5,000
[ 2 ] P 5,000 -‐ P 9,999
[ 3 ]P10,000 -‐ P 19,999
[ 4 ]P20,000 or more

Use a transi.on statement, i.e., an
introductory phrase to explain why the
next ques8ons are asked so that
respondents retain a sense of the logical
flow of the ques8onnaire.

“The past ques8ons asked about some of
your socio-‐demographic characteris8cs,
now, the next ques8ons will inquire
about your opinions about the skin care
system in the country…”

Use filter ques.ons if some ques8ons are not
applicable to all respondents

1. Are you currently prac8cing as a skin specialist?
Yes/No
If your answer is “No”, proceed to ques%on 3.
2. How long have you been prac8cing as a skin
specialist?____
3. What factors have hindered your prac8ce?

Filter ques8ons should not require extensive page
flipping by the respondents or memory of
answers to earlier ques8ons.

9. Sequencing of ques6ons
•  Straigh{orward and easy to answer ques8ons

at the beginning of ques8onnaire.
•  Complex and difficult ques8ons placed
towards the middle of the ques8onnaire.
•  Personal and sensi8ve ques8ons towards the
end of the ques8onnaire
Never start a ques8onnaire with an open
ques8on that requires a lot of wri8ng or
explaining.

10. Do a cover le]er explaining the purpose and
mechanics of the ques8onnaire, and ethical issues
•  Introduce self as interviewer or inves8gator

•  Tell what the research is all about, its purpose and why
respondent’s par8cipa8on is important,
•  Say how informa8on will be asked and how long it will take
to complete
•  Assure confiden8ality of responses, privacy, anonymity, how
results will be used
•  State whom to contact if have any queries, who to return
filled ques8onnaires to and date by which should be
returned
•  Thank recipient for their help
•  Signature, name and 8tle.

Include an informed consent form
The objec8ves and mechanics of the survey was

explained to me and I understood it well. I also
understand that my par8cipa8on is voluntary and I
can withdraw from the survey aker having agreed to
par8cipate. I am free to refuse to answer any
ques8on that is asked in the ques8onnaire.

I hereby provide INFORMED CONSENT to take part in
this study en8tled….
Par8cipant signature: Date:
Witness: Sign:

12. Pilot tes6ng

Pre-‐tes8ng or pilo8ng of the ques8ons
is not op8onal — it is crucial.

If 8me or money does not permit
pilot-‐tes8ng of the ques8onnaire,
then the study should not be
conducted at all.

Pilot-‐tes8ng can be used to sharpen the
focus of ques8ons, structure certain
responses, eliminate irrelevant, vague
ques8ons, and can be of great help in
revising the wording of ques8ons.
•  paper color
•  Font
•  8ming of each sec8on of the
ques8onnaire

Instrumentation PDF

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Instrumentation PDF

Transféré par

Droits d'auteur :

Formats disponibles

INSTRUMENTATION

• Use exis8ng relevant ques8onnaires

– asking too many ques8ons, the result of

• Ques8onnaire data must be able to provide

– The speciﬁc research objec8ves must

ques6ons ques6ons ques6ons ques6ons ques6ons ques6ons

• Category-­‐ Only one response can be selected

• Ra6ng scales -­‐A ra8ng device is used to record

• How many 8mes a month do you go to a

• Should not ask respondent to rank more

3. Items can be derived from exploratory interviews with

4. It should follow strictly the rules for wri8ng ques8ons

1. Discrete, nominal scales

• A. Visual analogue scales

• Do they feel comfortable answering the

• Is the wording of items clear?

• Do the respondents draw the same meaning

1. To reduce number of items

Qualita8ve item analysis

4. Cronbach’s coeﬃcient (for scales with mul8ple response

2. Are all domains covered by at least 5 items (content

1. Concurrent validity-­‐ How well do the results

2. Predic8ve validity-­‐ How well do the results

Example: Do treated subjects score be]er than untreated

2. Do the scores change with exposure to the

4. Are the results consistent with theory or social/clinical

correlate A1, Atotal r=0.29

DOMAINS (N=39) ITEM-TOTAL ALPHA

B. Affect 0.52-0.87 0.9003

D. Community beliefs 0.61-0.87 0.8607

E. Personality (n=6) 0.65-0.84 0.8514

1. If item is “bad” in all methods, delete item

2. If item is “bad” in frequency endorsement and inter-­‐

3. If item is “bad” in item-­‐total and Cronbach’s alpha,

1. If items are internally consistent, then a

2. To interpret, index score can be treated as a

• Straigh{orward and easy to answer ques8ons

• Introduce self as interviewer or inves8gator

The objec8ves and mechanics of the survey was

Vous aimerez peut-être aussi

•  Use exis8ng relevant ques8onnaires

– asking too many ques8ons, the result of

•  Ques8onnaire data must be able to provide

– The speciﬁc research objec8ves must

•  Category-‐ Only one response can be selected

•  Ra6ng scales -‐A ra8ng device is used to record

•  How many 8mes a month do you go to a

•  Should not ask respondent to rank more

3.  Items can be derived from exploratory interviews with

4.  It should follow strictly the rules for wri8ng ques8ons

1.  Discrete, nominal scales

•  A. Visual analogue scales

•  Do they feel comfortable answering the

•  Is the wording of items clear?

•  Do the respondents draw the same meaning

1.  To reduce number of items

1. Concurrent validity-‐ How well do the results

2. Predic8ve validity-‐ How well do the results

2.  Do the scores change with exposure to the

4.  Are the results consistent with theory or social/clinical

1.  If item is “bad” in all methods, delete item

2.  If item is “bad” in frequency endorsement and inter-‐

3.  If item is “bad” in item-‐total and Cronbach’s alpha,

1.  If items are internally consistent, then a

•  Straigh{orward and easy to answer ques8ons

•  Introduce self as interviewer or inves8gator