Class 13

Marketing Analytics
CBC, Sarah, COPD, and CBC

Stevens and
Darden09

Charles
book club

How to do Response Modeling

1. Test something using n names. Keep track
of Xs and RESPONSE (1/0).
2. Use the n names to build a model that
predicts RESPONSE.
3. Use that model to score new names (for
which you know the Xs).
4. Mail to the top scoring names.
Where to draw the cut depends on the
economics.

1. Test something using n names

2. Use the n names to build a
model to predict response.
We tested our
mailing on
4,000 names.
We have
several X
variables.
We will use
regression to
forecast
FLORENCE.

Regression of Florence on
Related Purchase
SUMMARY OUTPUT
Regression Statistics
Multiple R
0.107248523
R Square
0.011502246
Adjusted R Square
0.011172527
Standard Error
0.266713758
Observations
3000

Forecast Score =
0.057 +
0.0235*Related
Purchase

ANOVA
df
Regression
Residual
Total

Intercept
Related Purchase

1
2998
2999

SS
MS
F
2.481586485 2.481586485 34.88498803
213.2664135 0.071136229
215.748

Coefficients Standard Error

t Stat
P-value
0.057089558 0.006020467 9.482579519 4.87592E-21
0.023486083 0.003976411 5.906351499 3.89041E-09

High t
and low
p!

3. Use the model to score the

new names.
High scores
mean likely
to buy
Florence.

4. Mail to top scoring names.

Mail if score > .1 because we need at least a
.1 response rate to make money given cost
=$1 and response is worth $10
Because the model is so simple, this is the
same as mailing to all those with related
purchases > =2.

Mailing to
Related
Purchases >=2
achieves $214

Cap One Product Design

Lets here what you did!

How you Did

Click on the team names below to view more detailed results.

Team Name:

Rounds

Solicit.&
Devlop
Cost:

Cost of
Pieces
Mailed:

Total Mailing Total No. of

Costs:
Responses

Total
Response
Value

Total Profit:

Score

27-May

2 $15,000.00 $375,000.00 $391,600.00

12936 $756,404.00 $364,804.00

100

Team Eldrick
Pepe
Nepveux

2 $21,000.00 $218,600.00 $241,200.00

17867 $522,874.00 $281,674.00

77.2

2 $18,000.00 $375,000.00 $394,600.00

9270 $652,714.00 $258,114.00

70.8

Nanners

2 $21,000.00

$47,500.00

$70,100.00

2949

$23,508.00

60.0

Tiger

1 $16,000.00

$6.00

$16,806.00

$0.00 ($16,806.00)

60.0

$93,608.00

TEST all cells with 4K

$21,000.00

$375,000.00

$397,600.00 22380

$1,159,555.00

$761,955.00

TEST all cells with 3K

$21,000.00

$375,000.00

$397,600.00 22904

$1,188,169.00

$790,569.00

TEST all cells with 2K

$21,000.00

$375,000.00

$397,600.00 17376

$990,948.00

$593,348.00

TEST all cells with 1K

$21,000.00

$375,000.00

$397,600.00 19542

$965,827.00

$568,227.00

Cap One Product Design

Dont rely on regression of exhibit 2 data.
Things have changed.
BK score is an average

Test most cells and roll out the HIGHEST

VALUE cell in each column.
Total value is responses*their value
You should not ignore the fact that the value of response
depends on the cell.
If you tested all cells equally, roll out the cell in each
column that created the most value.

TESTING STRATEGIES
Test only the best
Very Risky
Case Data are not that relevant
the environment has changed
only AVERAGE BK score is available

Design an Experiment
Test a carefully selected subset of cells
Use the results to build a model to forecast all 36 cells
Roll out the cells with best forecasted profit

TEST all 36 Cells--roll out the best testing cells

the safest strategy.

Death Wish Marketing

The failure to develop and test several
marketing options is a form of death
wish marketing

Clancey and Krieg, Counter-Intuitive Marketing, NY: Free Press, 2000 (quoted in Lynn
and Lynn, Experiments and Quasi-Experiments: Tools for Evaluating Marketing
Options, WP No. 03-18-03, The center for Hospitality Research.)

In his memoirs, David Ogilvy says he succeeding in

advertising because he was always ready to run a few
ads he deemed to be losers. Invariably, some were
big hits, leading him to revise his theories. (Russo and
Schoemaker, Decision Traps)

Change the mindset

Ask How would we test this?
Ask, why not test this?
Get excited about testing it!

They test, why dont you?

Dance with Chance, Makridakis, Hogarth, and Gaba

In 90s Swedish doctors implanted 81 pace

makers...but only turned half of them on!
Every patient experienced improvement.

1,103 heart attack victims given the potent

drug..2,789 given a placebo
20% death rate for drug, 21% for placebo.
In both groups, those who were diligent with
their meds lived longer than those who did
not.

Government by chance
Supercrunchers, Ian Ayers

Piggyback on other Random Processes

NH kids applying to magnet schools were
chosen by lottery to attend.
Thats all we need to test the efficacy of magnet
schools! (p 73)

Since 1998 in India, 1/3 of villages were

assigned a female chief (Pradhan) at random

Sarah Gets a Diamond Exercise

6,000
Diamonds

in the
training
set
3,142
Diamonds

in the
test set

Advice for Sarah

Use ln(price) as your dependent variable
Create a column labeled lnprice using =ln(). Thereafter think of
this new variable as your dependent variable.
Convert your forecasts of ln(Price) back to prices by using =exp().
Be sure you do this before sending me your price forecasts

Use ln(carat weight) as a predictor variable

Use either numbers (1 to 5?) or sets of dummy variables
for the other characteristics.
Consider using several regression models..not just one for
all diamonds.

What we did
Pepe and Nanners both used the ln ln model
and got a mape of 20.7
Team Eldrick included numerical values for
the other Cs and did much better?
Team EDI is a professional data mining
firm.

Regression Accounts for correlated

Xs
Average
Price

Count

$11,541.53
$13,127.33
$11,484.70
$9,326.66
$5,886.18

253
2,482
2,428
708
129

$15,255.78
$11,539.19
$12,712.24
$12,520.05
$10,487.35
$8,989.64

661
778
1,013
1,501
1,079
968

FL
IF
VVS1
VVS2
VS1
VS2
SI1

$63,776.00
$22,105.84
$16,845.68
$14,142.18
$13,694.11
$11,809.05
$8,018.86

4
219
285
666
1,192
1,575
2,059

TOTAL

$11,791.58

6,000

CUT
Signature-Ideal
Ideal
Very Good
Good
Fair

COLOR
D
E
F
G
H
I

CLARITY

Why do SI
diamonds
have lower
avg price than
Ideal cut
diamonds?

Because sig ideal are

likely to be smaller.
Regression can
handle this?

Stevens Sarah Results

Pepe

"May27"

Team
Eldrick

3142

3142

3142

3142

3142

MAPE

20.07%

33.17%

9.41%

21.94%

20.07%

SCORE

80

75

100

80

80

TEAM

TIGER

Nanners

Eldrick gets the 100. Pepe, Tiger,

nanners all did about the same.
May27 had a technical error.

Colonial Broadcasting Company

Please read the case
Any questions about the case?

Use the regression results to

answer these questions
Which of the three networks had the highest
rated TV movies in 1992?
Regression 1 tells us that ABN had an average
rating of 13.363+1.397 = 14.76

What was the 1992 average rating of TV

movies from CBC?
Regression 1 says 13.363!

Regression with dummy

variables goes thru the group
averages.

Use the regression results to

answer these questions
Conventional Wisdom says that FACT based
movies do better. What do the data tell us?
Regression 2 tells us that FACT movies beat
FICTION movies by 1.4 points (on average) in
1992.

How strong is the evidence?

The result is statistically significant. The t was 2.6
and the p was .01. It did not happen by chance.

Use the regression results to

answer these questions
If we expect 1993 results to be similar to those
in 1992, what are the chances that a randomly
chosen CBC TV movie will get a rating greater
than 15?
Regression 1 says a CBC rating will have mean
13.363 and standard deviation of 2.42.
Probability the rating will be less than 15 is
NORMDIST(15,13.363,2.42,true) = 0.750.
The probability the rating will greater than 15 is
0.25.

Use the regression results to

answer these questions
Regression 2 says that FACT based movies are rated
higher by 1.4 points (on average).
Regression 3 says that FACT movies are rated 1.8
points higher (on average).
What the heck is going on?
FACT and STARS are correlated in our data.
FACT movies had either more or fewer STARS (on
average) than FICTION movies.
Since STARS improve the rating, then FACT based
movies must have had fewer STARS..that explains why
FACT beat FICTION by only 1.4. For a given number
of STARS, FACT beats fiction by 1.8.

Use the regression results to

answer these questions
If we know whether the movie is FACT and
how many STARS it has, does it also help
to know (if we are trying to predict the
rating) the competition rating?
YES. The t for COMPETION in regression 4
is -2.3 and the p is 0.03.
The negative sign just means that the higher the
competition the lower is the rating expected to
be. That makes total sense.

Final QUIZ
930 to 1130. Thursday, April 29.
Open book and notes.
No searching the internet or each other for
help with specific questions.
All material used in the course is usable.
Ill gladly give a help sessionjust let me
know when and where and how many.