Business Research Method

Experimentation

An Introduction to Experimental Design Statistical studies can be classified as being either experimental or observational. In an experimental study, one or more factors( Independent Variables) are controlled so that data can be obtained about how the factors influence the variables of interest(Dependent Variable). In an observational study, no attempt is made to control the factors. Cause-and-effect relationships are easier to establish in experimental studies than in

 Experiment: A research method in which conditions are controlled so that one or more independent variables can be manipulated and their effect on dependent variable is measured . To study impact of a sports coach on cricket teams performance an experiment can be conducted

Experiment
. Experiment:

A research method in which conditions are controlled so that one or more independent variables can be manipulated and their effect on dependent variable is measured . To study impact of a sports coach on cricket teams performance an experiment can be conducted

Experiment
.-A research investigation in which conditions

are controlled -One independent variable is manipulated (sometimes more than one) -Its effect on a dependent variable is measured -To test a hypothesis

Some Definitions
Absolute Experiment: If we want to know impact of a sports coach on cricket teams performance it is an absolute experiment Comparative experiment :If we want to know impact of one sports coach on cricket teams performance compared to impact of another sports coach on cricket teams performance it is a comparative experiment Experimental units: Subjects or entities subjected to various treatments are called experimental units

Some Definitions
- Experimental group: The group of subjects exposed to an experimental treatment. Group of students provided with a sport coach - Control Group :A group of subjects who are exposed to the control conditions in an experiment, that is they are subjects not exposed to the experimental treatment Group of students not provided with a sports coach

Treatment

- Treatment: It refers to conditions to which experimental groups are subjected and control groups are not subjected Provision of sports coach

Extraneous Variable

Extraneous variable: Independent variables that are not directly linked with the study but may influence dependent variable: If the study wants to test a hypothesis that a relationship exists between academic achievement and self concept of children then academic achievement is dependent and self concept is independent variable. Intelligence may also affect academic achievement but it is not related to studys purpose and hence it is an extraneous variable.

Experimental Design
Experimental Design is a set of procedures specifying --Manipulation of the Independent Variable --Selection of Dependent Variable --Experimental groups & Control groups --Control Over Extraneous Variables

Manipulation of independent Variable The experimenter has some degree of control over the independent variable The variable is independent because its value can be manipulated by the experimenter. To test relationship between package design & sales Expose customers to three package designs A,B &C. These packs are placed on shelves of select outlets Consumers response is measured Package design is independent variable ,which is manipulated & there are 3 treatment levels A,B & C

Selection of Dependent Variable

Selection of dependent variable is another important decision Sales volume of the product is considered as dependent variable in package design example Selecting dependent variable may not be easy in all cases To evaluate effectiveness of various ad programmes dependent variable can be brand image, brand awareness & product sales Purpose of the research helps in selection of appropriate dependent variable

Experimental & Control Groups

Experimental group is a group of test units exposed to change in independent variable Control group is a group of test units not exposed to change in independent variable In package design example a group of supermarkets (Experimental group) is selected & each package design is displayed for a month Another group of supermarkets (Control group) continue to carry regular package design for that period Sales of product are measured for both groups & difference in sales is analysed to find out impact of design on sales

Controlling Extraneous Variables Extraneous (Confounding/Mediating) variables need to be controlled otherwise it is difficult to find out whether the change in dependent variable is due to change in independent variable or change in extraneous variable

Ways to control extraneous variable Randomization Hold Conditions Constant Matching Subjects Design Control Statistical Control

Randomisation
Process of assigning test units randomly to experimental treatments and assigning experimental treatments randomly to test units It helps to spread effects of extraneous variable equally over the test units Most popular method Not effective for small samples Department stores assigned randomly to Experimental & Control groups If 3 versions of test commercials are to be assigned to 3 groups then one commercial is selected at random & assigned to a randomly selected group & process is repeated

Physical Control of Extraneous Variables

Hold Conditions Constant: The level of extraneous variables is kept constant throughout the experiment -- If price is the extraneous variable prices are not changed during experimental period

Physical Control of Extraneous Variables Matching Subjects: Judgmental sampling is used to assign test units to both experimental group & control group so that both groups are matched in terms of characteristics of test units ---Department stores are matched on the basis of annual sales, size and location. One store from each matched pair would be assigned to each experimental group

Statistical Control
Extraneous variables affecting dependent variable are identified & measured using appropriate statistical tools like ANOVA. Then effects of extraneous variable on dependent variable are removed by statistical adjustments If 5 departmental stores are used to study sales of products before & after in- house sales promotion then impact of the characteristics of the stores can be removed through use of ANOVA-randomised block design using department stores as blocks

Design Control
Selecting appropriate experimental designs to conduct experiments helps in controlling specific extraneous variables . Experimental Designs are of 4 types -Pre Experimental -True Experimental -Quasi experimental -Statistical

Experimental Validity
Validity: Extent to which a research process is accurate & reflects actual market conditions Internal Validity: It measure the extent of change in dependent variable in an experiment caused by manipulation of independent variable External Validity :It measures to what extent inferences derived from experiment can be generalised to real environment

Internal Validity
Control of extraneous variable is a necessary condition for establishing internal validity Internal validity is the basic minimum requirement of an experiment Six major types of extraneous variables are threats to internal validity

Factors Influencing Internal Validity

History Maturation Testing Instrumentation Selection Mortality

Type of Extraneous Variable History - Specific events in the environment between the Before and After measurement that are historic in nature (rare occurrence)

Example

Maturation - Subjects change during the course of the experiment due to passage of time. Subjects become tired Testing - The Before measure alerts or sensitizes subject to nature of experiment or second measure.

While test marketing new farm equipment area gets affected by floods Impact of promotional campaign for a car gets affected by steep price rise in petrol by Iraq war Drug trials carried over longer period of time results in some effect on account of physiological changes Questionnaire about the traditional role of women triggers enhanced awareness

Instrument - Changes in instrument to minmise testing effect result in response bias

New questions about women are interpreted differently from earlier questions. Heavy users in experimental group & moderate & low users in control group

Selection test units selected for experimental group differ from those assigned to control group

Mortality - Sample attrition; some subjects withdraw from experiment

Subjects in TV viewing study drop out before completion

External Validity Experiments conducted in natural settings offer more external validity as compared to experiments in controlled conditions Since generalisation is easier with field conditions

Experimental Environment
Laboratory Environment: Experiments conducted in controlled conditions Showing ads or products to select consumers in controlled conditions and blind taste tests Advantages Effects of extraneous variables can be minimised -History effects eliminated -Provides more internal validity -time & cost effective -reduces the risk of information about products & ideas being passed on to competitors Disadvantages -Expts conducted in artificial conditions &results may not hold well in actual conditions -less external validity -results influenced by testing effects ,Test units aware of being tested & may not respond naturally

Field Environment
Experiments conducted in natural settings --Launching products in select regions --Observing consumer behaviour regarding a POP display in supermarkets --analysing customer response to trial offers Advantages -high degree of external validity Disadvantages -Low degree of internal validity -Require greater time& efforts and are expensive

Laboratory Experiment Artificial-Low Realism Few Extraneous Variables High control Low Cost Short Duration Subjects Aware of Participation

Field Experiment Natural-High Realism Many Extraneous Variables Low control High Cost Long Duration Subjects Unaware of Participation

Different Basic Experimental Designs

Pre-Experimental Designs True Experimental Designs Quasi Experimental Designs Statistical Designs

Notations
X: Treatment or manipulation of independent variable O: Observation or measurement of dependent variable R: Random assignment of test units to experimental groups EG : Experimental group exposed to experimental treatment CG :Control group not exposed to experimental treatment Notations in the diagram are read from left to right

Conventions
Movement from left to right indicates movement through time Horizontal alignment of symbols implies that all those symbols refer to specific treatment group Vertical alignment of symbol implies that those symbols refer to activities or events that occur simultaneously

Examples
X O1 O2 means that a given group of test units was exposed to treatment variable X and the response was measured at two different points in time ,O1 and O2 R X1 O1 R X2 O2 means that two groups of test units were randomly assigned to two different treatment groups at the same time and the dependent variable was measured in two groups simultaneously

Pre-Experimental Designs
---They lack proper control mechanism to deal with effects of extraneous variables One Shot Design (After Only) One Group Pretest-Posttest Static Group Design

One Shot Design (After Only)

It involves exposing experimental group to treatment X after which the measurement (O1) of dependent variable is taken . EG : X O1 To measure effectiveness of a test commercial TV programme containing test Ad (X) is selected Telephone interviews are conducted of respondents who report having watched the TV programme previous night Respondents are asked whether they recall having seen the Ad. Measurements of Dependent variable are recalls (O1) Recalls are compared with norm scores for interpretation.

One Shot Design (After Only)

Drawbacks No random assignment of test units Extraneous variables are not controlled Internal validity low There is no other measurement against which O1 can be compared (what would happen if X was absent) More appropriate for exploratory studies and not conclusive studies

 Exposing an experimental group to experimental treatment (X) and taking measurements of dependent variable before(O1) and after(O2) the treatment EG: O1 X O2 Difference between O1 and O2 measures the impact of treatment on dependent variable To measure effectiveness of a test commercial Respondents are recruited on the basis of convenience Administered a personal interview to measure attitude towards the product Then they watch a TV programme containing Commercial After watching TV programme they are again administered a personal interview to measure attitude towards product Effectiveness of test commercial is measured as O2-O1 Influenced by maturation,testing ,mortality, selection bias affecting IV

One Group Pretest -Posttest

Static Group Design

Two groups of test units are involved EG &CG EG is exposed to treatment CG is not exposed Measurements are taken for both groups after Expt. EG: X O1 CG: O2 Difference between O1& O2 is effect of treatment To test effectiveness of TV commercial Two groups of respondents are recruited EG is exposed to TV programme containing Ad Attitude towards product of both EG &CG are measured Effectiveness of test Ad is measured as O1-O2 Selection bias ,mortality affects results with less IV

True Experimental Designs

---Use randomisation & one or more control groups to reduce effect of extraneous variables Pre test Post test Control Group Design Post test only Control Group Design Solomon Four Group design

Pretest-Posttest Control Group Design Two groups of test units EG &CG are considered Test units are assigned to these 2 groups randomly Pre test measurements of dependent variable are taken for both groups EG is exposed to treatment Post test measurements of dependent variable are taken for both groups EG: R O1 X O2 CG: R O3 O4 Treatment Effect (TE) is (O2-O1)-(O4-O3)

Pre Test Post Test Control Group Design

To measure effectiveness of a TV commercial Sample of respondents is selected at random Half of these would be randomly assigned to EG & other half would form CG Both groups would be administered a questionnaire to obtain pre-test measurement on attitude towards the product Only EG would be exposed to TV programme containing commercial Then respondents in both groups would be administered a questionnaire to obtain post test measurements on attitude towards product Effectiveness measure is (O2-O1)-(O4-O3) This design provides accurate results & addresses most of extraneous variables accept testing effects

Posttest test only Control Group Design Both EG & CG are formed by random assignment of test units EG is exposed to experimental treatment CG is not Post test measurement of dependent variable is taken for both the groups EG: R X O1 CG: R O2 Treatment Effect (TE) equals O1-O2

Posttest Only Control Group Design

To measure effectiveness of a TV commercial Sample of respondents is selected at random Half of these would be randomly assigned to EG & other half would form CG Only EG would be exposed to TV programme containing commercial Then respondents in both groups would be administered a questionnaire to obtain post test measurements on attitude towards product Difference in attitudes of EG &CG is used as effectiveness measure

Posttest Only Control Group Design Involves only 2 groups & one measurement Significant advantage in terms of time ,cost & sample size Selection bias & mortality which are the major drawbacks can be controlled through carefully designed experimental procedures Because of its simplicity it is most popular in marketing design

Solomon Four Group Design

It involves 4 groups ,2 EGs & 2 CGs Six measurements are taken 2 pretest &4 posttest Also known as 4 group 6 study design EG: R O1 X O2 CG: R O3 O4 EG: R X O5 CG: R O6 Design provides various observations which can be analysed O2-O1; O4-O3; O2-O4; O5-O6 Addresses all extraneous variables but is expensive & consumes time & efforts

Quasi Experimental Designs

Are used when randomisation is not possible & timing of test presentation lacks control of researcher Not as effective as true designs but better than pre-design Prominent is Time Series Design A series of measurements are taken on dependent variable for a group of test units before & after the experimental treatment O1,O2,O3 X O4, O5, O6 Trends before treatment are compared with trends after treatment to determine effectiveness of treatment Aids in identifying permanent change & temporary change Threats on account of history & instrumentation

Quasi Experimental Designs

To test effectiveness of a test commercial Broadcasting test commercial a predetermined number of times & examining data from a pre-existing test panel Scheduling of test commercial can be controlled but it is uncertain when or whether panel members are exposed to it Purchases of panel members before ,during and after the campaign are examined to find out whether test commercial has a short term effect ,long term effect or no effect To find out impact of price change on sales of a product, a series of observations are taken before the price change &after the price change. Trends before treatment are compared with trends after treatment to determine change

Multiple Time Series Design

Similar to time series except that another group of test units is added to serve as control group EG:O1,O2,O3 X O4,O5,O6 CG:O7,O8,O9 X O10,O11,O12 Test commercial would be shown only in a few of test cities Panel members in these cities would form EG Panel members in the cities where commercial was not shown would constitute control group

One-Shot Design Internal Validity Problems

History
weak

Instrumentation
not relevant

Maturation
weak

Selection
weak

Testing
not relevant

Mortality
weak

One-Group Pretest-Posttest Internal Validity Problems

History
weak

Instrumentation
weak

Maturation
weak

Selection
controlled

Testing
weak

Mortality
controlled

Static-Group Design Internal Validity Problems

History
controlled

Instrumentation
controlled

Maturation
possible source of concern

Selection
weak

Testing
controlled

Mortality
weak

Pretest-Posttest Control Internal Validity Problems

History
controlled

Instrumentation
controlled

Maturation
controlled

Selection
controlled

Testing
controlled

Mortality
controlled

Posttest-Only Control Internal Validity Problems

History
controlled

Instrumentation
controlled

Maturation
controlled

Selection
controlled

Testing
controlled

Mortality
controlled

Solomon Four-Group Design Internal Validity Problems

History
controlled

Instrumentation
controlled

Maturation
controlled

Selection
controlled

Testing
controlled

Mortality
controlled

Statistical Designs
Aids in measuring effect of more than one independent variable Helps in isolating effects of most extraneous variables Four prominent designs in this category -Completely Randomised -Randomised Block -Latin Square -Factorial

Advanced Experimental Designs are More Complex

Completely randomized Randomized block design Latin square Factorial

Completely Randomized Design An experimental design that uses a random process to assign subjects (test units) and treatments to investigate the effects of only one independent variable (factor). There are n test units & k treatments (levels of factor) The n test units are assigned to k treatments randomly. Post test measurements are evaluated

Completely Randomized Design EG1: R X1 O1 EG2: R X2 O2 EG3: R X3 O3 EG1,EG2,EG3 are 3 experimental groups exposed to three experimental treatments X1,X2,X3 3 test cities are randomly exposed to 3 test commercials Then measurements are taken on sales in these 3 cities Differences in sales in 3 cities are analyzed to see

Example: Home Products, Inc. Home Products, Inc. is considering marketing a long-lasting car wax. Three different waxes (Type 1, Type 2,and Type 3) have been developed. In order to test the durability of these waxes, 5 new cars were waxed with Type 1, 5 with Type 2, and 5 with Type 3. Each car was then repeatedly run through an automatic carwash until the wax coating showed signs of deterioration. The number of times each car went through the carwash is shown on the next slide. Home Products, Inc. must decide which wax to market. Are the three waxes equally effective?

Example: Home Products, Inc.

Observation 1 2 3 4 5 Sample Mean 55 Sample Variance WAX Type 1 48 54 57 54 62 68 26.0 26.5 WAX Type 2 73 63 66 64 74 57 24.5 WAX Type 3 51 63 61 54 56

Example: Home Products, Inc.

Completely Randomized Design ANOVA Table Source of Variation Treatments Error Total Sum of Squares 490 308 798 Degrees of Freedom 2 12 14 Mean Squares

245 9.55 25.667

To evaluate efficacy of a weight loss drug A sample of 40 consumers is selected Assigned randomly to two treatment levels 25 to treatment 1 & 15 to treatment 2 Consumers in group 1 are asked to take drug for one month Consumers in group 2 are not given any drug After experiment measurements are taken for both groups & differences if any are analysed to find out effectiveness of the drug Applicable when test units are homogeneous Extraneous variables can be controlled A single variable is evaluated

Completely Randomized Design

Randomized Block Design

An extension of the completely randomized design in which a single extraneous variable that might affect test units response to the treatment has been identified and the effects of this variable are isolated by blocking out its effects.

Example Randomised Block Design

A consumer product company wants to examine impact of price change on sales of its newly developed health drink Store type is extraneous variable which would also affect product sales Company has decided to conduct experiment using 3 price levels & 3 store types Nine retail outlets are segregated according to store type Treatment levels (prices) are assigned randomly to each test unit ( retail Outlet)

Example Randomised Block Design

Price Change Drug Store Rs 10 Rs 20 Rs 30

Supermarkets Malls

Example Randomised Block Design

Using this design two effects are determined :Main effect & Block Effect Main effect is direct effect of price change on sales: It is the average effect of a particular treatment on dependent variable regardless of extraneous variable Block effect refers to influence of extraneous variable on product sales. It is the influence of store type on the product sales. This is obtained by determining customer response to each price change for each store type

Factorial Design
An experiment that investigates the interaction of two or more independent variables on a single dependent variable.

Factorial Design
A 3x2 design has two factors first with 3 levels & second with 2 levels A 3x3 design has 2 factors each with 3 levels A 2x2x2 design has 3 factors each with 2 levels A 3x2x4 design has 3 factors with 3,2 7 4 levels respectively

Factorial Design
Researcher is investigating believability of Ads on 0-100 scale Two magazine Ads A & B are to be compared Gender of the reader is another factor This is a 2x2 factorial experiment Permits to test 3 hypothesis Which Ad is more believable (Main) Which gender tends to believe magazine Ads more(Main) Which gender finds which ad more believable(Interaction)

2 x 2 Factorial Design
Ad A Men Women
70 60

Ad B 65
Main Effects > of Gender

65

Main Effects of Ad

>

100 90 80 70
Believability

Interaction Between Gender and Advertising Copy

Wo me n

60 50 40 30 20 10 Ad A

Men

Ad B

Factorial Design -- Roller Skates

Package Design Price $25 $30 $35 Red Cell 1 Cell 2 Cell 3 Gold Cell 4 Cell 5 Cell 6

Latin Square Design

A balanced, two-way classification scheme that attempts to control or block out the effect of two or more extraneous factors by restricting randomization with respect to the row and column effects.

Latin Square Example

To find out impact of three different Ads on sales Pricing & income levels of consumers are EV In latin square levels of all the 3 variables must be same ( 2EV and 1 INDEP) A table is then developed with levels of one EV representing rows ,levels of second EV representing columns . Levels of independent variable (treatments) are exposed to each cell on a random basis so that each row has all treatments & each column has all treatments Treatment effect is determined for each cell Analysis shows which treatment level influences dependent variable more

Latin Square Example

Pricing Levels Rs 10000 Rs 12000 Rs 14000

Low Income Middle Income Ad-B Ad-C Ad-A Ad-A Ad-B Ad-C

High Income Ad-C Ad-A Ad-B

Latin Square
Interaction effects are assumed to be minimal or absent Number of treatment levels for both confounding factors must be equal Suppose a retail grocery chain wishes to control for shelf space & city where product is sold It markets in only 3 cities & wishes to experiment with 4 levels of shelf space Having unequal number of levels for each factor will eliminate latin square as a possibility

TEST MARKETING
Not just trying something out But scientific testing

Controlled experimentation

Test Marketing
An experimental procedure that provides an opportunity to test a new product or a new marketing plan under realistic market conditions to measure sales or profit potential.

Selecting a Test Market

Population size Demographic composition Lifestyle considerations Competitive situation Media Self-contained trading area Secrecy

Control Method of Test Marketing

Small city Low chance of being detected Distribution is forced (guaranteed)