Vous êtes sur la page 1sur 5

The Case for Incrementality

How to Measure the Real ROI of Your Marketing Programs


By Marc Solomon, Senior Strategist, Catalyst

Page 1

2012 Catalyst

The Case for Incrementality

s an analyst and a bit of a data geek, Im excited by the continuing trend of marketers using data to guide overall strategy and specific decisions. New technology makes it easier and cheaper to gather, store and manage Big Data. Of course, all of this data is useful only if it provides true insights that are actionable, and most marketers are not fully or even correctly leveraging their data. A recent CEB (Corporate Executive Board) study provides some interesting perspective on this front. Harvard Business Review summarized the findings in a blog post, concluding that the vast majority of marketers still rely too much on intuitionwhile the few who do use data aggressively for the most part do it badly. For marketers, new CRM (Customer Relationship Management) tools make it easier not only to track individual sales, but also track the reach of online and offline campaigns. And the real value comes from connecting the dots so you can see at the customer level how much your target audience is buying right? Well, thats only partially correct. What really matters is not how much your target audience bought, but how much more they bought because of your marketing.

Incremental vs. Total Sales


Lets compare the results of two hypothetical campaigns. Each required $100,000 of marketing investment. Your CRM data tells you that the customers you targeted with Campaign A generated $500,000 in sales, and the customers in Campaign B generated $250,000. So you might conclude that the first campaign is the clear winner. It generated double the sales, didnt it? Not necessarily. What if Campaign A was targeted to your best customers, who would have spent $500,000 even without your marketing? This means, then, that the $100,000 marketing spend actually did nothing for you. And, what if Campaign B was targeted to prospects who would not have otherwise purchased anything if you hadnt invested your $100,000 to market to them? This is starting to look like a pretty nice return on your investment, and youre probably now much more excited about Campaign B.

Which Campaign Delivered Greater ROI?


Campaign A Campaign B

Marketing Spend Actual Sales (with marketing) Expected Sales (no marketing) Incremental Sales (due to marketing)

$100,000 $500,000 $500,000 $0

$100,000 $250,000 $0 $250,000

Page 2

2012 Catalyst

The Case for Incrementality

This is an extreme example to make a point: You should be focused on incremental, not total, sales when evaluating campaign impact. Put another way, its important to focus on causation, not just correlation. Its not enough to know that customers you targeted made purchases (correlation); you want to know if the marketing drove those purchases (causation). The difference between correlation and causation is important, and explained nicely on the Adobe Digital Marketing Blog.

Creating a Baseline
The tricky part in calculating incremental sales is figuring out the difference between sales levels with and without marketing. How do you know what customers would have bought in an alternate reality where they didnt receive any marketing? You can use historic trends to estimate baseline business as usual sales youd expect without new marketing, but this is imprecise at best and, could be massively wrong. There are so many factors that impact salesproduct/pricing changes, economy, competition, weather, etc.that it is nearly impossible to predict sales with high accuracy. A much better approach is to set up a controlled experimenta concept that may bring back memories of 10th grade science class. You can get a refresher at Wikipedia: Controlled Experiment. The key point is to isolate the factor that youre trying to measurein this case, the impact of the marketing campaignand hold constant all other factors. To do this, you need to randomly pick some of your target audience to be held out from the campaign as a control or baseline group. Its critical that this group looks just like the customers who will receive the marketing campaign, so that you can be confident that any difference in sales between the groups was driven by the campaign and not some underlying difference between the customers or their market environment. In the example below, you can see a $3 difference between the test and control groups, which can be attributed to the marketing campaign.

Sales per Customer

Page 3

2012 Catalyst

The Case for Incrementality

Isolating Channel Effectiveness This approach can be leveraged not only to measure the impact of an overall campaign but also the impact of specific campaign channels or tactics. In the case below, this enables distinct measurement of both the direct mail impact and email impact.

Sales per Customer

Keep in mind that the same basic principle of randomly assigning customers to each of the cells is critical; otherwise you dont have a fair apples-to-apples comparison. It is tempting and, in some ways logical, to use the email channel for all customers who have provided email addresses and appropriate permissions. However, this introduces a bias to the marketing test. The customers who provided email addresses are unlikely to be the same as those who have not. They probably are different demographically, and more importantly, they may have a different attitude toward their relationship with you. If you ran a test in which you sent emails to everyone you could, then compared their sales to customers you didnt email, you would not be able to determine how much of the difference was driven by the emails and how much was explained by underlying differences between the customers who did/didnt grant you the ability to email.

Page 4

2012 Catalyst

The Case for Incrementality

Practical Considerations
While measuring incremental sales in this way is the theoretically right way to assess marketing impact, the reality is that this isnt always viable or worth the effort. It works extremely well with direct marketing (online and offline) but is very difficult with mass marketing/advertising, where its impossible to create an identical random control group. It is typically worthwhile when there are large target audiences and there is minimal opportunity cost to holding out a statistically significant control group. But it may not be worthwhile if you need to hold out a large portion of your audience from any marketing for the sole purpose of robust measurement. For example, in the above case about isolating the impact of email campaigns, you may decide that you dont need a robust measurement of email impact. Since emails are cheap to send and presumably have some positive impact, you may not want to bother creating and managing a holdout population for the sake of measurement.

...its important to remember that its the incremental, not total, sales that are important to determining your marketing ROI.
However, even in cases where a robust controlled experiment isnt possible or isnt worth the effort, its important to remember that its the incremental, not total, sales that are important to determining your marketing ROI. And if you cant measure incrementality directly, then its a good time to resort to an estimate based on historic trends and forecasts. While that wont be perfect and could be quite wrong, it will still give you a better sense of your marketing impact than looking purely at the total sales figures.

About the Author Marc Solomon, who holds an MBA from the Stanford Graduate School of Business, has leveraged analytics to drive results for Fortune 500 companies for nearly 20 years. As senior strategist, Marc uses his keen understanding of key market trends and insight into target audience behaviors and attitudes to provide strategic recommendations. Hes the former vice president of consumer marketing and analytics at Opower, vice president of marketing and analysis at Capital One, and a consultant with Booz-Allen & Hamilton.

800.836.7720 | www.catalystinc.com |

Facebook |

Twitter |

LinkedIn

Page 5

2012 Catalyst

Vous aimerez peut-être aussi