Lecture #6

Data Collection Tools

Questionnaire Design

By the end of this lecture, student should be able to:

Differentiate between primary and secondary data
Identify different tools for data collection
Design a well structured questionnaire
Define validity and reliability of a data collection tool

Data collection is essential for answering a research question. In research, there are different
methods used to gather information, all of which fall into two categories, i.e. primary data, and
secondary data.
Primary Data:
Primary data is data originated for the first time by the researcher through direct efforts and
experience, specifically for the purpose of addressing his research problem. It is also known as
the first hand or raw data.
Primary data collection is quite expensive, as the research is conducted by the organization or
agency itself, which requires resources like investment and manpower. The data collection is
under direct control and supervision of the investigator.
The data can be collected through various methods like surveys, observations, physical testing,
mailed questionnaires, questionnaire filled and sent by enumerators, personal interviews,
telephonic interviews, focus groups, case studies, etc.

Secondary Data:
Secondary data implies second-hand information which is already collected and recorded by any
person other than the user for a purpose not related to the current research problem. It is the
readily available form of data collected from various sources like censuses, government
publications, and internal records of the organization, reports, books, journal articles, and
websites and so on.
Secondary data offer several advantages as it is easily available, saves time and cost of the
researcher. But there are some disadvantages as the data is gathered for the purposes other than
the problem in mind, so the usefulness of the data may be limited in a number of ways like
relevance and accuracy. Moreover, the objective and the method adopted for acquiring data may
not be suitable to the current situation. Therefore, before using secondary data, these factors
should be kept in mind.

1
Comparison between primary and secondary data:
Basis for
Primary data Secondary data
comparison
Primary data refers to the first hand
Secondary data means data collected
Meaning data gathered by the researcher
by someone else earlier.
himself.
Data Real time data Past data
Process Very involved Quick and easy
Government publications, websites,
Surveys, observations, experiments,
Source books, journal articles, internal records
questionnaire, personal interview, etc.
etc.
Cost effectiveness Expensive Economical
Collection time Long Short
Always specific to the researcher's May or may not be specific to the
Specific
needs. researcher's need.
Available in Crude form Refined form
Accuracy and
More Relatively less
Reliability

Examples for data collection tools:

Checklists:
Checklists structure a persons observation or evaluation of a certain performance. They can be
simple lists of criteria that can be marked as present or absent, or can provide space for observer
comments.

2
Example of a checklist for hand hygiene assessment
OBSERVATION #1 #2 #3
Hand hygiene technique with soap and water
1. Wet hand with water using temperature that is comfortable.
2. Apply appropriate soap, and wash hands and wrists vigorously for 15 seconds,
covering all surfaces of hands and fingers. (No bar soap.)
3. Rinse hands under running water, allowing water to drip from fingertips.
4. Thoroughly dry hands and wrists with paper towel.
5. Turn faucet off using a dry paper towel to touch the handle, protecting your clean
hands from the contaminated handle.
Hand hygiene technique for waterless sanitizer
1. Apply product to palm of one hand.
2. Rub hands together covering all surfaces of hands and fingers until hands are dry.
3. Fingernails (Direct patient care givers)
Fingernails (Direct patient care givers)
1. No artificial nails or enhancements.
2. Natural nails are short (1/4 inch) and well groomed.
3. Hands, including the area around nails, free of inflammation.
4. Nail polish is in good repair.

Describe corrective action for any item not performed properly.

Interviews:
In-Depth Interviews include both individual interviews (e.g., one-on-one) as well as group
interviews (including focus groups). The data can be recorded in a wide variety of ways
including audio recording, video recording or written notes.

Questionnaires:
Instruments used for collecting data in survey research. They usually include a set of questions
that explore a specific topic and collect information about demographics, opinions, attitudes, or
behaviors. They could be self-completed (self-administered) or through an interview (interview
questionnaire).

Case Report From (CRF) and worksheets are tools for organizing and reporting data in
medical research as laboratory and radiological tests, clinical examination and others like patient
reported outcomes: physical functioning or pain assessment.

Criteria of a data collection tool:

3
How can a researcher be sure that the data gathering instrument being used will measure what it
is supposed to measure and will do this in a consistent manner? This is a question that can only
be answered by examining the validity and reliability of a research instrument.

Validity:
Validity can be defined as the degree to which a test measures what it is supposed to measure.
There are three basic approaches to the validity of tests and measures. These are content validity,
construct validity, and criterion-related validity.
Content Validity: sometimes called logical or rational validity, is the estimate of how much a
measure represents every single element of a construct. Face validity is the extent to which a test
is subjectively viewed as covering the concept it aims to measure.
Construct Validity: This type of validity is a judgment based on the accumulation of evidence
from numerous studies using a specific measuring instrument. Evaluation of construct validity
requires examining the relationship of the measure being evaluated with variables known to be
related or theoretically related to the construct measured by the instrument. If a researcher is
trying to measure back pain in a group of patients, he should make sure that his tool is measuring
pain not numbness, discomfort, anxiety or any other symptom.
Criterion-Related Validity: This type of validity provides evidence about how well scores on the
new measure correlate with other measures of the same construct or very similar underlying
constructs that theoretically should be related. One of the simplest ways to assess criterion
related validity is to compare it to a known standard. When a newly developed tool is compared
to a Gold Standard instrument (Concurrent). It also refers to the degree to which a test can
predict a behaviour in the future (Predictive).

Reliability:
The reliability of a research instrument concerns the extent to which the instrument yields the
same results on repeated trials. Reliability estimates are used to evaluate:
a) The stability of measures administered at different times to the same individuals or using the
same standard (testretest reliability)
b) The equivalence of sets of items from the same test (internal consistency) or of different
observers scoring a behavior or event using the same instrument (inter-rater reliability).
Questionnaire design
Questionnaire is an important data collection tool. For some items, we have no other tool to
measure what we want to know, examples include headache, pain, tolerance to pain and diet.

Constructing a Questionnaire:
When constructing a questionnaire one should consider the following points:
1. Formulate objectives

4
 2. Review necessary information
3. List the variables to be collected
4. Formulate questions (preliminary draft), translate
5. Test the questionnaire (pilot study)
6. Revise and redesign

Types of questions:
1- Open-ended questions allow respondents to answer using their own words. They are useful
in exploring the motives, attitudes and beliefs. They are difficult to code and need special
methods of analysis. Example: What habits do you believe increase a persons chance of having a
heart attack?

2- Closed-ended questions are appropriate if the range of possible responses is already known.
They are easier to code and analyse, but if the responses do not cover all, or at least the critical
possibilities, then the study findings may be distorted. Example: What do you think about the
standard of service you received at the family health care centre? Excellent 1 Very good 2
Good 3 Bad 4

The Dichotomous answer (Yes-No, Agree-Disagree, Approve-Disapprove)

The simplest form, and the easiest to analyze, is the dichotomous response question (Yes-No,
Agree-Disagree, Approve-Disapprove). However, this type of questions may be less informative
and it is not appropriate for investigating issues in depth, for sensitive topics or for attitudinal
questions.

Likert scale:
A Likert scale is a type of psychometric response scale often used in questionnaires, and is the
most widely used scale in survey research for attitudes. When responding to a Likert
questionnaire item, respondents specify their level of agreement to a statement. Likert scaling is
a bipolar scaling method, measuring either positive or negative response to a statement. The
scale is named after Rensis Likert, who published a report describing its use in 1932.
Example: Do you think that the patient should be offered all the options for the treatment of his
conditions to choose from?
1. Strongly disagree
2. Disagree
3. Neither agrees nor disagrees
4. Agree
5. Strongly agree

General rules for constructing a questionnaire:

5
1. A questionnaire is developed to address certain research issue(s) and provide information on
the specific research questions; only items that contribute to this purpose should be included.
2. Most health questionnaires start with some basic socio-demographic questions, such as age,
gender, education, job, marital status, number and gender of children, income among other items
used for descriptive purposes as part of the statistical analysis.
3. The questionnaire should start with easy, non-threatening questions. Questions dealing with
sensitive issues should be delayed to near the end of the questionnaire after obtaining the
confidence of the respondent.
4. The questions should be ordered so that they flow logically and enable the respondent to
follow and understand.
5. The questionnaire may be self-administered, e.g., given to the respondent to fill by himself.
This method is cheaper and quick but suitable only with well educated and motivated
respondents, e.g., doctors and teachers. Self-administered questionnaire may have a relatively
low-response rate (some will not complete it) but it is free from interviewer bias. If respondents
or their majority are expected to be illiterate, the questionnaire is filled by an interviewer(s). This
is called interview-questionnaire and standardization of how the question should be delivered
becomes an issue to avoid interviewer bias.
6. Questions that investigate how a respondent thinks about an issue (opinions, beliefs values and
attitudes) are more difficult to formulate because they attempt to tap into issues that are often
complex and may be sensitive.

Errors in Questionnaire Design

There are 2 types of errors in designing a questionnaire:
1. Errors in Question Wording
Leading questions: Using leading words that suggest that there is a more desirable answer.
For example: how often do you attend health education sessions classes? Such questions are
likely to lead respondents to report, or over-report, attendance as the wording assumed that
all respondents attend and that only the frequency of attendance has to be ascertained.
Respondents who do not attend may be reluctant to admit this. Example: You dont sleep
well, Do you? Yes No . Correct: Do you sleep well? Yes No
Emotionally loaded words or phases: The use of strong harsh or emotive words may distort
responses. For example: Do you consider smoking a bad/dirty habit? Yes-No In this
example the use of the word dirty is likely to increase an affirmative response.
Questions that assume knowledge of technical terminology: For example: Use yellow
coloured skin and sclera instead of jaundice.
Double-barrelled questions: For Example: Have you had nausea, vomiting or abdominal
colic since your last visit? Yes-No. It is not clear whether a Yes response refers to any or all
of the manifestations mentioned in the question head. If distinction is important, two or three
questions should be asked

6
 Use of negatives: For example: Are you against not smoking in public places? Yes- No.
Negative should be avoided, as they are liable to make the question confusing and hence
difficult to answer.
Vague, imprecise or ambiguous questions: For example: Do you know many elderly
people? Yes-No The term elderly may be interpreted differently by respondents,
depending on their own age group. Also, words such as many (often, occasionally,
regularly, enough), should be avoided as they refer to imprecise quantity, meaning different
things to different people.
2. Errors in Construction of Response Categories
(a) Providing an incomplete list of response categories
What is your current marital status?

In this example, the category widowed is omitted.

(b) Response categories are not mutually exclusive
What is your age?

(c) Response categories are not mutually exclusive

What are the elements of Diabetes management explained to you by the physician?

7
The categories are not mutually exclusive, as the physician may recommend more than one line
of management.

(d) Response categories are not logically ordered

How many diarrheal attacks do you have per day?

(e) Instructions are inappropriate

What do you understand by SARS? (Tick only one)
Viral disease ( )
Pandemic ( )
Fatality rate 50% ( )
There is valid vaccination ( )
There is more than one correct answer; therefore "tick only one" is an inappropriate instruction.

(f) Inconsistent layout of response categories

Confusing layout of the response; Both vertical and horizontal ordering at the same time
1. Yes/No
2. Yes No
3. Yes ( ) No ( )
4. Yes ( ) No ( )
5. Dont know ( )
(g) Use of abbreviations
They should be written in full or they may confuse the respondent.
Example:
o "Y/N" should be written as "Yes/No",
o "DK" as "Dont know"
o "N/A" as "not applicable" ,
o "PA" as" per annum"
o "< >" less than or more than.

8
 (h) Not providing the opportunity to respond "Yes or No" before continuing with
subsequent parts of the question
What type of contraception are you using?
Pills ( ) IUD ( ) Local ( ) Injections ( ) Others ( ) No ( )

This question should be reconstructed in 2 parts as:

a- Do you currently use any contraceptive method? Yes ( ) No ( )
b- If yes, what type of contraception are you using?
Pills ( ) IUD ( ) local ( ) injections ( ) others ( )

The goal is to collect data that is:

Valid: measures the quantity or concept that is supposed to be measured
Reliable: measures the quantity or concept in a consistent or reproducible manner
Unbiased: measures the quantity or concept in a way that does not systematically under-
or overestimate the true value
Discriminating: can distinguish adequately between respondents for whom the
underlying level of the quantity or concept is different