Week 3

Primary Data Collection:

Survey Research
Observation Method
Experimental Research
By Dr. Muhammad Ramzan
mramzaninfo@gmail.com,
03004487844
Edited by Ahsan Khan Eco
ahsankhaneco@yahoo.com
03008046243

Survey Research
To collect primary data, specifically for a
project in hand.
Surveys provide a quick, inexpensive, efficient
and accurate means of assessing information
about a population
Identifying characteristics of a target market,
measuring customer attitude and describing
consumer purchase patterns etc.

Survey Method
The survey method is based on the questioning of
respondents regarding their behaviour, intentions,
attitudes, awareness, motivations, demographic and
lifestyle characteristics. These questions may be
verbally, in writing, or via computer and responses
may be obtained in any of these forms.
Typically the questioning is structured and a sample of
respondents are asked to select from a pre-determined
set of alternatives.
Example:
Disagree
Agree
Shopping at malls is fun 1
2
3
4
5
3

Types of Surveys

Structured versus unstructured

structured-formal standardised questions used
unstructured -informal, no standardised questions
degree of structure influences choice of media
Disguised versus undisguised
disguise - the concealing of purpose or sponsorship of
a study until completion
undisguised - respondent is aware of purpose and
sponsor of research
4

Advantages of Survey Methods

accommodate large sample sizes; increases
generalisability of results
standardisation all respondents react to questions
worded identically; response options/scales are same
administrative ease much simpler than a focus group/
in-depth interview; development of questionnaire is a
more complex process than the administration
ability of tapping into factors & relationships that are
not directly observable ie attitudes, feelings, preferences
tabulation and statistical analysis of data is easy
subgroup differences can be easily determined
5

Disadvantage of Survey Methods

difficulty of developing accurate survey
instruments (questionnaire design)
limits to the in-depth detail of data
lack of control over timeliness, & potential
low response rates
difficulties to determine if respondents are
responding truthfully
misinterpretation of data results &
inappropriate use of data analysis procedures
6

A Classification of Survey Methods-Mode of Administration

Survey
Methods

Telephone

Personal

In-Home

Traditional
Telephone

Mall
Intercept

Computer-Assisted
Telephone
Interviewing

Mail

Computer-Assisted
Personal Interviewing

Mail
Interview

Electronic

E-mail

Mail
Panel

Internet

Mail Methods
Can be conducted through mailing questionnaires to
pre-selected respondents via ordinary mail or mail
panel. It includes outgoing envelope, cover letter,
questionnaire, return envelope, the purpose, assurance
of anonymity, an appeal for completion and return
within due date.
There is no verbal interaction between researcher and
the respondent
Before stat of mail survey, respondents need to be
broadly identified, a valid mailing list, closely related to
population of interest is required
8

Mail Survey Advantages & Disadvantages

no interviewer bias, can respond at leisure
high geographic flexibility, more confidential information
cost confined to mailing list, forms and postage
follow up easy but time consuming
low response rate, high cost per response, long time delays, no
probing/observation; can read entire questionnaire before
answering which may influence response; higher possibility
misunderstand/clarification
never sure that the target respondent actually answered the
questions
complex questions not responded to well; open-ended questions
do not achieve lengthy written responses

Increasing Response Rates for Mail Surveys

Effective cover letter

Authorities help/reference letter
Incentives helps
Interesting questions
Use of peers/colleagues/professional association
Follow-ups
Advanced notification
Survey sponsorship
Keying questionnaires

Personal Methods
Personal In-home Interviews; respondents are interviewed
face-to-face in their homes. Interviewer contact respondents ask
questions and record responses
Mall-Intercept: Respondents are intercepted while they are
shopping in malls/superstores and brought to test facilities in the
malls. Researcher administers a questionnaire. Methods is more
efficient and becoming popular
Computer Assisted Personal Interviewing: Respondent sits in
front of a computer terminal and answer questions on the
computer screen using mouse and keyboard. User-friendly
electronic packages are available to design easily understandable
questions. Interviewer remains available as a host to guide the
respondent

Personal Methods Advantages & Disadvantages

persuasion/cooperation; better response rate; use observation
can use visual material; longer questionnaires
Can assist to interviewee; more precise selection of sample
more versatile questioning; extended probing possible
if self-completed, respondent can complete when convenient
travel time & expenses high, people reluctant to talk to strangers
interviewers presence may bias, inhibited, difficult to supervise and
control; difficult to recruit capable interviewers; geographic
flexibility limited
call back/follow up difficult; no anonymity of interviewees
Self-completed disadvantages similar to mail
12

Telephone Methods
Traditional Telephone Interview involves phoning
and asking a series of questions to respondents.
Researcher uses paper questionnaire and records
responses with a pencil
Computer-Assisted Telephone Interviewing:
Researcher wears a mini-hand set, reads questions from
compute screen and records responses directly in the
computer memory. In this method computer replaces
pencil and paper questionnaire

Telephone Method Advantages & Disadvantages

dialling efficient especially call backs; travel avoided
respondent does not have to open door to strangers
geographic coverage good; supervision and training excellent;
speed of data collection
less interviewer bias; greater anonymity of respondents
no observation; visuals excluded; retaining attention more
difficult therefore shorter questionnaires; only homes with
telephones; cost can be a factor with STD calls
there is limited interview times pertaining to the best times for
making and gaining responses
Interviewer is unable to control the interview; the respondent
can hang up at any time

Electronic Methods
Email Interviews require list of email addresses.
Survey is written within the body of email message.
Respondents type answers to close ended/open ended
questions at designated places and reply through email.
It requires data entry.
Internet Interviews are posted on a website using
HTML. Respondents need to open the website and
complete the survey. Respondents are sent links usually
through email. (Survey Monkey, Google Doc Forms)

Electronic Methods Advantages & Disadvantages

speed of response/ data collection
easy of development and testing
little cost of administration
easy to administer
the possibility of data corruption via virus transmission
possible unreliability of e-lists
are respondents representative of the population
possibility of bogus replies
16

A Comparative Evaluation of Survey Methods

Criteria

MallIn-Home
Intercept
Interviews

CAPI

Mail
Surveys

Mail E-Mail
Panels

Moderate
tohigh

High

Interviews
High
Moderate
tohigh

Diversityofquestions

Low

High

High

High

Moderate Moderate Moderate

Useofphysicalstimuli

Low

High

High

Moderate Moderate

Moderate
Low
Moderate

Moderate
tohigh
Potentially
high
Moderate
tohigh
Low
High
High

Perceivedanonymityof
therespondent
Socialdesirability

Moderate

Low

Moderate

High

High

Obtainingsensitive
information
Potentialforinterviewer
bias
Speed

High

Low

Low

Moderate

High

High

High

Moderate

Cost

Moderate

High

Flexibilityofdata
collection

Samplecontrol
Controlofdatacollection
environment
Controloffieldforce
Quantityofdata
Responserate

Phone

Moderate
tohigh
Moderate

Moderate Moderate
High

High

Low

Low
Low

Low

Moderate
tohigh
Low

Low

Low
Low
Low

Internet
Moderate
tohigh
Moderate
tohigh
Moderate
Lowto
moderate
Low

Moderate Moderate
High
High
High
High
Moderate Moderate Moderate
High
Moderate Moderate
High
High
Low
Moderate
Low
Very
Low
Low
Low
High
High
Moderate
High
Moderate
toHigh
Lowto
moderate
Low

Moderate Moderate
tohigh
tohigh
Moderate Moderate
tohigh
tohigh

Low
High
None
Low
Low

Low

Moderate

Low

Moderate Moderate
toHigh
None
None

High

Lowto
moderate
Lowto
moderate

High
Low

None
Very
high
Low

Factors determining choice of survey method

Researchers resources & objectives
Time horizon
Budget
Desired quality of data
Required level of generalization
Completeness & precision
Respondent characteristics
Willingness to participate
Ability to participate
Diversity of respondents
Type of questions asked
Complexity
Topic sensitivity
Amount of information required per respondent

18

There is no best form of

survey; each has advantages
and disadvantages.

Selected Questions to Determine the

Appropriate Technique
Is the assistance of an interviewer necessary?
Are respondents interested in the issues being
investigated?
Will cooperation be easily attained?
How quickly is the information needed?
Will the study require a long and complex
questionnaire?
How much is the budget?

Observation Method
It involves recording the behavioral pattern of people,
objects, and events in a systematic manner to obtain
information about a phenomenon of interest.
The observer does not question or communicate with the
people being observed.
Information obtained may be structured or unstructured,
direct or indirect and may be conducted in a natural or
contrived environment

Structured vs Unstructured Observation

For structured observation, the researcher
specifies in detail what is to be observed and how
the measurements are to be recorded, e.g., an
auditor performing inventory analysis in a store.
In unstructured observation, the observer
monitors all aspects of the phenomenon that
seem relevant to the problem at hand, e.g.,
observing children playing with new toys.

Disguised vs Undisguised Observation

In disguised observation, the respondents are
unaware that they are being observed.
Disguise may be accomplished by using oneway mirrors, hidden cameras, or unnoticeable
mechanical devices. Observers may be
disguised as shoppers or sales clerks.
In undisguised observation, the respondents
are aware that they are under observation.

Natural vs Contrived Observation

Natural observation involves observing
behavior as it takes places in the environment.
For example, one could observe the behavior
of respondents eating fast food in Burger
King.
In contrived (artificial) observation,
respondents' behavior is observed in an
artificial environment, such as a test kitchen.

A Classification of Observation Methods

Observation Methods

Personal
Observation

Mechanical
Observation

Audit

Content
Analysis

Trace
Analysis

Personal Observation
A researcher observes actual behavior as it
occurs.
The observer does not attempt to manipulate
the phenomenon being observed but merely
records what takes place.
For example, a researcher might record traffic
counts and observe traffic flows in a
department store.

Mechanical Observation
Mechanical devices rather than human observers record the
phenomenon.
Do not require respondents' direct participation.
Audimeter (attached to a television to continuously record what
channel that set is tuned. People meters attempted to measure the
channel to which TV is tuned but also who is watching)
Turnstiles records number of people entering or leaving a building.
On-site cameras (still, motion picture, or video)
Optical scanners in supermarkets
Do require respondent involvement.
eye-tracking monitors; psychogalvanometers (skin response)
voice pitch analyzers (emotional reactions through changes in
respondents voice
devices measuring response latency (amount of time it takes to
respond

Observation Methods: Audit

The researcher collects data by examining physical
records or performing inventory analysis.
Data are collected personally by the researcher.
The data are based upon counts, usually of physical
objects.
Retail and wholesale audits conducted by marketing
research suppliers

Observation Methods: Content Analysis

The objective, systematic, and quantitative
description of the manifest content of a
communication.
The unit of analysis may be words, characters
(individuals or objects), themes (propositions),
space and time measures (length or duration of the
message), or topics (subject of the message).
Analytical categories for classifying the units are
developed and the communication is broken down
according to prescribed rules.

Observation Methods: Trace Analysis

Data collection is based on physical traces, or evidence, of past
behavior.
The number of different fingerprints on a page was used to gauge
the readership of various advertisements in a magazine.
The position of the radio dials in cars brought in for service was
used to estimate share of listening audience of various radio
stations.
The age and condition of cars in a parking lot were used to assess
the affluence of customers.
The magazines people donated to charity were used to determine
people's favorite magazines.
Internet visitors leave traces which can be analyzed to examine
browsing and usage behavior by using cookies.

A Comparative Evaluation of Observation Methods

Criteria
Degree of structure
Degree of disguise
Ability to observe
in natural setting
Observation bias
Analysis Bias
General remarks

Personal
Mechanical
Observation Observation

Audit
Content
Analysis Analysis

Low
Low to high
Medium Low to high
High
Low to high

High
Low
High

High
High
Medium

High
High

Low
Low

Medium
Low

Low
Low to
Medium
Most
Can be
flexible intrusive

Expensive

Limited to
communications

Trace
Analysis
Medium
High
Low
Medium
Medium
Method of
last resort

Relative Advantages of Observation

They permit measurement of actual behavior rather
than reports of intended or preferred behavior.

There is no reporting bias, and potential bias caused

by the interviewer and the interviewing process is
eliminated or reduced.
Certain types of data can be collected only by
observation.
If the observed phenomenon occurs frequently or is
of short duration, observational methods may be
cheaper and faster than survey methods.

Relative Disadvantages of Observation

The reasons for the observed behavior may not be
determined since little is known about the underlying
motives, beliefs, attitudes, and preferences.
Selective perception (bias in the researcher's perception)
can bias the data.
Observational data are often time-consuming and
expensive, and it is difficult to observe certain forms of
behavior.
In some cases, the use of observational methods may be
unethical, as in observing people without their knowledge
or consent.
It is best to view observation as a complement to survey
methods, rather than as being in competition with them.

Experiment Method
A research investigation in which conditions
are controlled so that causal relationship
among variables is determined
One or more independent variable are
manipulated while holding all other possible
independent variables to observe the effects on
dependent variable
IVs effect on a dependent variable is measured
Used to test a hypothesis

Use of experimental research

Experimentation is commonly used to
infer causal relationship
Causality is the relationship between an
event (the cause) and a second event
(the effect), where the second event is
understood as a consequence of the first
A. the relationship of cause and effect
B. the principle that nothing can happen
without being caused
C. causal agency or quality

Basic issues of experimental design

Manipulation of the independent variable

Selection of dependent variable
Assignment of subjects (or other test units)
Control over extraneous variables
The experimenter has some degree of control over
the independent variable.

The variable is independent because its value can

be manipulated by the experimenter to whatever
he or she wishes it to be

Definitions and Concepts

Independent variables are variables or alternatives that
are manipulated and whose effects are measured and
compared, e.g., price levels.
Test units are individuals, organizations, or other entities
whose response to the independent variables or treatments
is being examined, e.g., consumers or stores.
Dependent variables are the variables which measure the
effect of the independent variables on the test units, e.g.,
sales, profits, and market shares.
Extraneous variables are all variables other than the
independent variables that affect the response of the test
units, e.g., store size, store location, and competitive effort.

Independent Variable (IV)

The experimenter controls independent
variable.
The variables value can be
manipulated by the experimenters to
whatever they wish it to be.

Dependent Variable (DV)

Its value is expected to be dependent on
the experimenters manipulation
Criterion or standard by which the results
are judged
Selection
e.g... sales volume, awareness, recall,

Measurement

DVs are the outcome of interest to the

researcher and the decision maker

Example
Hypothesis: Changes in price influences sales
Price here is IV and sales DV and other
possible variables are extraneous variables

Example

Influence of brand name (IV) on consumers taste(DV)

perceptions
Experimenter manipulated whether consumers preferred drinks
of labeled or unlabelled bottles
One week respondents were given drink a pack of bottles with
label ABC. Next week respondents received another pack of
drinks with brand labels (Pepsi, Fanta, Shezan etc)
Researcher measured reactions to drinks after each tasting as the
drink was the same.
Therefore the differences observed in taste (DV) could only be
attributed to the difference in labeling (IV).

Participants preferred for branded drinks, so the

conclusion (inference) is that brand name does
influence consumers taste perception

How may an experimenter control for

extraneous variation?

Eliminate extraneous variables

Hold conditions constant
Randomization
Matching subjects

Experimental Design
An experimental design is a set of procedures specifying:

the test units and how these units are to be divided into
homogeneous (all the same) subsamples,

what independent variables or treatments are to be

manipulated,

what dependent variables are to be measured; and

how the extraneous variables are to be controlled.

Validity in Experimentation
A researcher has to draw valid conclusions of effects of
independent variables (internal validity) on the study group
(sample) and make valid generalization (external validity) on
larger population of interest
Internal validity refers to whether the manipulation of the
independent variables or treatments actually caused the
observed effects on the dependent variables. Control of
extraneous variables is a necessary condition for establishing
internal validity.
External validity refers to whether the cause-and-effect
relationships found in the experiment can be generalized. To
what populations, settings, times, independent variables and
dependent variables can the results be projected?

Extraneous variables
History refers to specific events that are external to the
experiment but occur at the same time as the experiment.
Maturation (MA) refers to changes in the test units
(sample) themselves that occur with the passage of time.
Testing effects are caused by the process of
experimentation. Typically, these are the effects on the
experiment of taking a measure on the dependent
variable before and after the presentation of the
treatment.
The main testing effect (MT) occurs when a prior
observation affects a latter observation.

Extraneous Variables
In the interactive testing effect (IT), a prior measurement
affects the test unit's response to the independent variable.
Instrumentation (I) refers to changes in the measuring
instrument, in the observers or in the scores themselves.
Statistical regression effects (SR) occur when test units
with extreme scores move closer to the average score during
the course of the experiment.
Selection bias (SB) refers to the improper assignment of test
units to treatment conditions.
Mortality (MO) refers to the loss of test units while the
experiment is in progress.

Controlling extraneous variables

Randomization refers to the random assignment of test
units to experimental groups by using random numbers.
Treatment conditions are also randomly assigned to
experimental groups.
Matching involves comparing test units on a set of key
background variables before assigning them to the
treatment conditions.
Statistical control involves measuring the extraneous
variables and adjusting for their effects through statistical
analysis.
Design control involves the use of experiments designed
to control specific extraneous variables.

Laboratory VS Field experiments

A laboratory environment is an artificial one, which the
researcher constructs with the desired conditions specific to
the experiment. An artificial environment of
experimentation in which the researcher constructs the
desired conditions. To measure the effectiveness of a test
commercial could be conducted in a laboratory environment
by showing the test commercial embedded in a TV program
to respondents in a test theater.
Field environment is an experiential location set in actual
market conditions. Such as running test commercials on
actual TV stations.

Laboratory vs field experiments

Factor

Laboratory

Field

Environment
Control
Reactive Error
Demand Artifacts
Internal Validity
External Validity
Time
Number of Units
Ease of Implementation
Cost

Artificial
High
High
High
High
Low
Short
Small
High

Realistic
Low
Low
Low
Low
High
Long
Large
Low
Low

High

Laboratory experiment

Field experiment

Artificial-Low Realism

Natural-High Realism

Few Extraneous
Variables

Many Extraneous
Variables

High control

Low control

Low Cost

High Cost

Short Duration

Long Duration

Subjects Aware of
Participation

Subjects Unaware of
Participation

Limitations of experimentation
Experiments can be time consuming, particularly if the
researcher is interested in measuring the long-term
effects.
Experiments are often expensive. The requirements of
experimental group, control group, and multiple
measurements significantly add to the cost of research.
Experiments can be difficult to administer. It may be
impossible to control for the effects of the extraneous
variables, particularly in a field environment.
Competitors may deliberately contaminate the results of
a field experiment.

Test marketing
It is an application of a controlled experiment done in
limited but carefully selected test market. It involves a
replication of the planned national marketing program for
a product in the test markets. Often market mix
(independent) variables are varied in test marketing and
the sales (dependent variable) are monitored so that an
appropriate national marketing strategy can be identified
To test market acceptance of the product
Test alternative levels of marketing mix variables.

Criteria for the selection of test markets

1) Be large enough to produce meaningful projections. They
should contain at least 2% of the potential actual population.
2) Be representative demographically and with respect to
product consumption behavior.
3) Be representative with respect to media usage and with
respect to competition.
4) Be relatively isolated in terms of media and physical
distribution.
5) Have normal historical development in the product class.
6) Have marketing research and auditing services available.
7) Not be over-tested.

Summary
The scientific notion of causality implies that we can never
prove that X causes Y
We can only infer that X is one of the causes of Y in that it
makes the occurrence of Y probable's.
Three conditions must be made (I) concomitant, which implies
that X and Y must vary together in a hypothesized way (II)
time order of occurrence of variables, which implies that X
must precede Y and (III) elimination of other possible causal
factors. Experiments provide the most convincing evidence of
these conditions.
An experiment is formed when one or more independent
variables are manipulated by the researcher and their effect on
one or more dependent variables is measured.

Conclusion
There is no best data collection technique;
each has its advantages and disadvantages.
The suitability and appropriateness of data
collection technique/method depends on
the purpose, scope of the research and
researchers resources.

THANK YOU
By Dr. Muhammad Ramzan
mramzaninfo@gmail.com,
03004487844
Edited by Ahsan Khan Eco
ahsankhaneco@yahoo.com
03008046243