Académique Documents
Professionnel Documents
Culture Documents
27 March 2011
Methodology: BrandLift studies
1 Ad displayed to user
2 Randomised control group
Facebook’s ad systems automatically
generate a small control group of users
called a “holdout” who are identical to those
targeted by your campaign, except that they
were randomly selected to not see your ads.
3 Example Poll
4 Results
A comprehensive PowerPoint deliverable highlighting
study results and brand effect metrics:
348 Control
Questions
Question 1: Which landmark is running the “Housemates Wanted”
campaign?
Question 2: What is the prize for the winners of the Sydney Opera House
“Housemates” promotion?
Significance Testing
Significance Testing
Asterisks next to point lifts denote statistically significant differences between two proportions. For
example:
If 50% of Exposed respondents recall Sydney Opera House’s advertising and 25% of Control
respondents recall Sydney Opera House’s advertising, is there a significant difference between
Exposed and Control respondents?
Statistical reliability allows for differences between groups of people. In this example,
Exposed respondents (50%) are significantly more likely to recall Sydney Opera House’s
advertising than Control respondents (25%) at a 90% confidence level. This would be
illustrated with an asterisk next to the point lift (+25 pt. Lift) to show the significant difference.
+25 pts*
Significant comparisons between groups are done at a 90% confidence level; a standard
measure of reliability used in market research. A 90% confidence level simply means that if we
conducted the same survey 100 times, the data would be within a certain number of
percentage points above or below the reported percentage 9 times out of 10.
Campaign Results
Summary of Findings
Key Findings
2 Not only did the higher-funnel campaign awareness increase by a significant 17 point
increment, but the lower-funnel message association metric increase was also significant at
12 point incremental
3 Across both question, this campaign resonated with all age groups except for 18-24 year
olds who were statistically flat for both questions
Campaign Awareness
Awareness of the SOH Housemates Wanted campaign increased by a
tremendous and significant 17 incremental percentage points
100%
80%
60%
52%
25%
+3pt s -3pt s +1pt s
20%
10% 9%
7% 7% 7%
6%
+17pt s * -17pt s *
0%
80%
60%
53%
50%
47%
44% Cont rol [N=348]
42%
40% 39% Exposed [N=354]
33%
31%
28% 28%
25% 24% 25%
20%
13%
40%
40% 38%
34%
30%
14%
11%
Notes on Answers
10% Some answers have been
8%
6% abbreviated. True
5%
answers were “A
+12pt s * -11pt s * -4pt s +1pt s +2pt s helicopter ride around
Sydney Harbour” “Tickets
0%
to the upcoming Madam
Your phot o on a wall Helicopt er Ride Ballet Ticket s Model Replica Not sure Butterfly Ballet”, and “A
[N=107] [N=200] [N=84] [N=41] [N=267] wooden model of the
House”
*Indicates a statistically significant increase or decrease. Question
What is the prize for the winners of the Sydney Opera House “Housemates”
Message Association – Demo Breakout
This question resonated most strongly with users 25-54, and there were
significant lifts across both males and females as well
50%
40%
36%
34%
30%
13%
10% 10%
10% 9%
8%
7%
5% 6%