1

Finale and Survey on the Spot

Paul Conforti, one of the owners of Finale, a Boston-area premium dessert restaurant, talked about why,
in early 2010, he decided to use Survey on the Spot to collect market research data:

Were using Survey on the Spot to give our guests a real time opportunity to provide us
with feedback on their experience in the restaurant. We basically decided to use this to
replace traditional mystery shopping at our restaurant. In the past, we would pay a
company to come in once a month and have a dining experience and then provide us with
a report on that experience and we would compare their experience to our standard
operating procedures and steps of service and determine if we were doing a good job.
Now we can have dozens of customers each month complete the survey based on their
actual experience and they can tell us whether or not were doing a good job. I think
getting information directly from the customer rather than from a company that we are
paying to act as a proxy for the customer is a much better way to do it.

The results to date from customers have been very positive for Finale, almost too positive (see
Exhibit 1 for the April 2010 summary report) and Conforti was concerned that these results
indicated there was no room for improvement; something he doubted. Therefore, he had asked
Felicity Klass, an independent market research analyst, to take a look at the data and provide
more in-depth analysis based on all of the data collected to date. Given the very positive data
results and the apparent lack of things to improve, he was beginning to question whether dropping
the mystery shopper visits had been a good business decision. At least the mystery shoppers
always led to suggestions.

Survey on the Spot

As described by Ken Kimmel, Co-Founder and President of Survey on the Spot, the mobile marketing
research application was developed based on practical experience:

The idea was generated 10 years ago. My partner [Geoff Palmer, Co-Founder and CEO]
is a serial entrepreneur in technology and has been through several ventures in the
technology side of the business. I was, at the time, the CMO of Dunkin Donuts. We
were standing on the street and he was showing me the latest cell phone gadget, which
was a camera. I said: Wouldnt it be great if I could get pictures of the Dunkin Donuts
stores and how the donut cases looked? We talked about it a little bit and the technology
wasnt where we wanted it.Fast forward 10 years and Geoff had bought an iPhone and
he says: You know, I think we can do that now...we took [our prototype] to some
potential clients and said: Does that make sense? and they said it made a lot of sense.
Then we went into the development of the system. We went live November 1, 2009 and
just launched our third revision of the application on the iPhone store.1 The target for
us is the current guest satisfaction business. Why did we pick the guest satisfaction
business? People spend an awful lot of money on it and the bar is reasonably low. [A

1
An Android version of the Survey on the Spot app was added in December, 2010.
 2

firm that] gets 30,000 transactions a month hopes to get 30 guest satisfaction surveys via
the web.

Survey on the Spot enabled clients to gain feedback directly from customers cell phones. Rather than
more traditional secret shoppers, comment cards, or Web surveys referenced from the bottom of the
transaction receipt, customers could report on their experience while still in the store or restaurant. The
application was able to provide: Surveys of all types including Customer Satisfaction Surveys, Research
Surveys, Polling Surveys, Public Opinion Surveys, Survey Panels, Menu Taste Panels, Conventions and
Special Events on mobile phones, iPhones, iPod Touch, smartphones and Android mobile phones.2
While any sort of survey could be used, the initial focus for the business had been real time customer
satisfaction results.

Using a small table tent, sticker or menu callout would make the customer aware that the restaurant used
Survey on the Spot. The customer could then download the app from the companys web site
(www.surveyonthespot.com) while still dining and potentially fill out a survey about their current
situation or their overall opinion of customer service or the purchase experience. If the client requested it,
the system sent management an alert based on the customers feedback. Based on specific survey
submission criteria, the Survey on the Spot alert function sent a real time email or a text message about a
service failure (or extraordinary service) and could have it go directly to the manager on site, the district
manager or even the CEO. According to Geoff Palmer:

That alert happens instantaneously. So on a scale of 1 to 10, and 1 is horrible, you can say
that for a 1, 2 or 3, I want to get alerted. You can do that multiple times for the same
survey, so that if you had ten questions and three of them were dissatisfied, the manager
gets 3 alerts or just one summary alert and that really gets their attention when they see
three things happened at the same location. If its someone up in higher management,
theyre going to call the store manager and say, Did you look at your email? You have a
problem. Thats something that any of the traditional methods cannot do because they are
all after the fact.

The alert service can prompt somebody to talk to somebody and help correct bad
behavior or pat them on the back if theyre doing a great job. We are getting an
equal amount of people saying, Oh, I had a great experience. So-and-so should be
the model.

Businesses could also provide incentives or rewards to encourage customer participation, which would
show up in a My Wallet icon on the Survey on the Spot screen. The reward might be a reduction off the
current meal, or a coupon for a future visit or any award deemed an appropriate incentive, with the added
appeal to the customer of having the coupon available on the mobile phone. The reward capability
created immediate engagement and the possibility of increasing the customers purchases on that visit.
Geoff commented:

2
www.surveyonthespot.com/faq/
 3

People are engaged with their mobile phones and since they are engaged, they may be
willing to do it [fill out a survey], especially with the fact that they can get a reward
instantly. The restaurants have the choice of making the reward applicable at a future
visit or $2 off a dessert that you might not have ordered, but maybe would at the same
time.

One behavioral change required by Survey on the Spot, was getting clients accustomed to the difference
in delivery method. As opposed to the standard web-based survey, a Survey on the Spot mobile survey
was comparatively short and clients may have to adjust to not getting a probability sample with
demographics. Ken Kimmel commented:

You cant do a 30 question survey on your phone. I would argue you probably cant even
do that on the web; its just too long. So [it requires] conditioning clients to be content
with shorter surveys, maybe not having all the demographics. I still, intellectually, have
trouble understanding when you have a survey, which people are self-selecting to take;
its not randomized at all.

Finale started using Survey on the Spot in January, 2010 and, by the end of the first quarter had
discontinued using the mystery shoppers it had previously employed once a month (at a cost of
$150 a visit per restaurant location) and had eliminated quarterly in-store surveys to each
location. Using Survey on the Spot, cost Finale $30 per month and, according to Paul was a no
brainer as a management decision.

Finale Desserterie & Bakery

Three Harvard Business School students started Finale Desserterie & Bakery in 1996. The
concept was upscale dining focused on creating a sensational dessert experience. The original
location was between the upscale shopping district of Newbury Street and the Theater District in
Boston, Massachusetts. Distinctive offerings 3 helped garner considerable positive press
coverage (articles in the Boston Globe, Pastry Art & Design and Bon Appetit, and inclusion on
the TV Food Network's "Best of Dessert" and "Best of Boston") and attracted dessert fans from
Boston and beyond. The company opened a second location in Harvard Square in 2002 and a
third outlet in Brookline's Coolidge Corner in July 2006. In addition to dessert, the restaurants
also offered dining selections and catering. Paul Conforti discussed the Finale customer profile:

Our typical customer is a young woman, above average education, above

average income, typically single and coming to Finale after dinner, the movies,
or the theater, as a chance to extend the evening, whether its on a date or with a
group of girlfriends. Thats our evening customer. Its kind of the standard
profile. The daytime customer is either an office worker or a person who lives
near the restaurant coming to us for a gourmet lunch: a soup, salad, sandwich

3
See Finale menus at www.finaledesserts.com/menus-viewmenus.php
 4

type lunch or a pastry and cake or sweet from our takeout bakery.
Demographically, its still, on average, the same profile, though a little more
diverse daytime customer group versus the evening customers.

With a stated mission To be the universally recognized brand of super-premium desserts, coupled with
a pledge that: We create sensational dessert experiences Finale continually sought to upgrade the total
dining experience. Using cutting-edge technology was part of the process. Paul Conforti explained both
aspects:

Since our customer is well educated and affluent and probably an early adopter of
technology, we like to be perceived as on the cutting edge of technology and provide
opportunities for the guests to use technology to interact with us. It helps us both in
knowing the way that technology is going and the immediacy of the feedback. We
actually have a chance to get the feedback and react to it on the spot. we get the alert
that the [Survey on the Spot] service offers. Managers get emails instantly when a survey
is completed and if the guest is in the building and we can identify them based on the
information theyve provided and if theres a need to react to a service problem, we can
actually do so real time.

There were additional, business-related reasons to use Survey on the Spot. Paul continued:

As I explained earlier, we like the idea of getting more data directly from
customers, rather than a single data point from a paid professional. Secondly,
theres a cost component to it. Its much more cost effective. Were paying less
to get more data, so thats a pretty easy decision to make. Theres the benefit of
receiving the information real time and being able to immediately run reports that
aggregate the information or dissect the information without having to go through
manual gyrations to compile it and repost it. We have the cost savings associated
with it [Survey on the Spot] versus the old mystery shopping program and the
benefit of what we believe is improved service because of the existence of the
program.

Prior to using Survey on the Spot guest satisfaction surveys, Finale conducted other types of
marketing research. In 2009, it had conducted several focus groups to evaluate its brand image.
From that input, it generated a follow-up survey garnering 2,000 responses from its email list of
over 15,000 addresses. In addition, Finale administered quarterly customer satisfaction surveys
handed out in the restaurants to ascertain demographics, brand awareness and location prior to
visiting Finale. All this research was in addition to the mystery shoppers, who reviewed the
customer experience against Finales performance standards.

Mystery shopping "enables managers and executives to measure how closely their customers'
experiences reflect what the company expects them to experience... The information provided
through mystery shopping allows businesses to precisely adjust operations to maximize the
customer experience. 4 For example, a mystery shopper might assess how many types of cream
cheese were on display in a bagel shop. A typical customer would only notice if their favorite
flavor was not there. 5 Conforti was not alone in looking for an alternative to mystery shopping,
4
www.mysteryshop.org/shoppers
5
Spielberg, 2005, page 6
 5

other restaurateurs such as the OCharleys chain, El Pollo Loco chain and Backyard Burgers
franchises were using or testing survey alternatives to mystery shoppers. 6

After starting Survey on the Spot, the mystery shopping reviews were terminated. The quarterly
surveys had also been discontinued due to concerns about possible respondent fatigue, leaving
Survey on the Spot as the sole technique tracking customer satisfaction. Since the sample size
for February was very small, in March servers were incentivized to encourage customers to
complete the survey. In addition, in March, the server name was added to the survey.

One interesting result from Finales use of Survey on the Spot was brought up by both Ken
Kimmel and Paul Conforti; the numbers were too good. Paul agreed:

Were getting really good scores, so we dont have a lot of instances where
recovery is required. It was a little unnerving at first. We thought maybe we
were getting only the people who were having a good experience and missing all
the people who were having a bad experience. Ken and Geoff told us: You dont
need to worry about that. We have plenty of examples of people getting bad
survey results.

There were some other concerns driven by the need to keep the survey short and doable
on the cell phone. Paul noted:

We cant get as much information as wed like. We dont collect the demographic
information anymore, which I find helpful in terms of trying to understand who
my customer is. The guest has to be technology literate in order complete it, so
you have some adverse or self selection because of that. We have the challenge
of making sure the customer is aware of their ability to complete the survey, but
you have that issue, no matter what. You obviously could have people gaming
the system where they get their friends and relatives to complete surveys that are
favorable but on the flip side theres the benefit of also collecting email
addresses from people who complete the surveys.

Survey on the Spot asked a dozen questions versus the mystery shopper survey of 100 questions.
(See Case Exhibits 1 and 3 for the Survey on the Spot Questions and a Sample of the Mystery
Shopper Questions.) There were gains and losses from the resulting information. Paul
commented:

It doesnt necessarily matter to me whether or not the handle of the coffee cup is
placed at 4:00, but our assessment of service says that the handle of the cup is
supposed to be placed at 4:00, so we get to know [from the secret shopper survey]
whether or not our standards are being followed. .on the one hand, Im a
process systems-oriented person, so I want information that tells me whether or
not were adhering to the process and systems weve established, but on the other
hand, the ultimate goal is simply to have happy customers and have them come
back and tell other people about it. So, from that perspective, were probably
6
Ibid.
 6

getting better information from Survey on the Spot. but in terms of monitoring
processes and systems, things we internally care about, but that the guest may not
even be aware of, we obviously lose that if we dont [mystery] shop. The reality
is, we were doing only one [mystery] shop a month anyway, so we didnt have a
statistically significant understanding whether we follow the procedures.

Depending on the needs of the client, Survey on the Spot could provide data in a number
of formats; with or without analysis (client data analysis could be arranged for an
additional fee beyond the normal monthly charge). Clients either saw the collected data,
individually or aggregated, in a variety of pie charts, graphs and tables, customized to
their needs. Survey on the Spot developed the report format for Finale and did not
conduct further analysis. Finales CEO, VP of Operations, Chef and the General
Managers of each store location had access to the monthly reports and to date used only
the summary analysis.

One useful aspect in reporting verbatims to the open-ended questions was the word
cloud. As respondents made statements in response to the survey questions, key words
and associations were stored. The client received a summary sheet of the most frequently
cited words and could access the related verbatim responses by clicking on the word.
(See Case Exhibit 1 for Word Cloud from Survey Question 6 and Exhibit 4 for Open-
Ended Verbatim responses based on desserts and prices.)

Although Finale had only been using Survey on the Spot for the first quarter of 2010,
results had already become apparent. Paul Conforti commented on what he was looking
for:

We are hoping that Survey on the Spot will get us better, faster, cheaper
information than mystery shopping ever could. We are in a guest-focused
business and Survey on the Spot provides us with a great opportunity to gather
information directly from the guest. Its my hope that our use of Survey on the
Spot will result in better service and it will provide us with information we can
use to improve ourselves and the experience we offer the guest.

As mentioned earlier, Conforti had asked Felicity Klass to do extra analysis and report
her findings (See Exhibit 4 for the report). Conforti now had the monthly summary report
and Klass data analysis report in front of him. He had to reconsider his decision about
the mystery shoppers as well as decide what the data indicated to him about the changes,
if any, that he should make at Finale to improve the restaurant experience?

In submitting this case to the Case Research Journal for widespread distribution in print and electronic media, we
certify that it is original work, based on real events in a real organization. It has not been published and is not under
review elsewhere. Copyright holders have given written permission for the use of any material not permitted by the
"Fair Use Doctrine." The host organization has assigned a release authorizing the publication of all information
gathered with understandings of confidentiality.
 7

7
Exhibit 1 : Finale April 2010 Survey on the Spot Results Report

Finale Guest Satisfaction

Start Date: April 1, 2010

End Date: April 30, 2010

1. What was your server's name? (It's included on your check)

Text Answers

2. How often do you visit Finale?

Results based on 98 responses to this question.

3. What did you order on this visit? (Check all that apply)
Results based on 98 responses to this question.

7
surveyonthespot.com
 8

4. Were you satisfied with your overall experience at Finale?

Results based on 98 responses to this question Average value: 9.03

5. How likely would you be to return to Finale?

Results based on 98 responses to this question. Average value: 9.29

6. What would encourage you to come to Finale more often?

Word Cloud

away i'm choices location special dessert great

lower cakes friendly things baby jill discounts

ambiance breaks treat aware bigger better god hot

good dinner deserts come price like sugar closer blog

 9

door allow bodies cloth service cherries sure specials carte

food based selection day area present server love cocktail

cake wine music menu chocolate place awful treats

online boston just change gay called order coupons california

college served shared don't free it's cheaper i'd

sorbets drinks fruit available bread changed prices

bakery need cheese cheescakes offers home options

clothing think desserts larger tapas finale branch

promotions small list section atmosphere

7. How satisfied were you with the service you received?

Results based on 97 responses to this question. Average value: 9.19

8. How satisfied were you with the cleanliness/appearance of the

Desserterie/Bakery Caf?
Results based on 97 responses to this question. Average value: 9.21
 10

9. Did your food and beverage selection meet your expectations?

Results based on 97 responses to this question. Average value: 8.86

10. Please explain the reason for your answer:

Word Cloud

like server plate carried tasty average love ingredients

better amazing great mousse presentation good

extremely ice balance selection perfect cale apple quick play

collapsed available absolutely answer came courteous

comprehensive bland berries bit able ordered anchor

service come beautiful concepts appealing appearance cause

 11

molten dessert palate nice fresh think desserts

delicious day crepes charming coupon best added big

food creme-brele cajeta considered just atmosphere spot bisque

bitter seven cutie hot caramels clean candle arranged

recommended wasn't friendly adventure gave drinks feel

decent got caramel dry little bathroom coat cake pasta

mode really coffee chocolate excellent lots

cheesecake expectations small complexity

11. Was your experience/food/beverage worth the price you

paid for it?
Results based on 97 responses to this question. Average value: 7.84

12. Would you recommend Finale to a friend?

Results based on 97 responses to this question. Net Promoter Score: 67%
 12

Exhibit 2 Sample Mystery Shopper Restaurant Questions 8

1. Were you acknowledged by anyone within 2 minutes of being seated?

2. Were you asked for a drink order at greeting?
3. Did the server initiate eye contact with guests when speaking?
4. Did the server compliment orders when taking them? (good choice or that is
excellent)
5. Did the food items match the menu descriptions?
6. Was wine opened and presented correctly when served?
7. Did the server place your handled cup/glass at your right side with the handle at 4?
8. Were food items served within 20 minutes of being ordered?
9. Was the check presented within 2-3 minutes of request?
10. Was the air comfortable in temperature?
11. Were the windows and door glass clean in all areas?
12. Was the floor clean of all paper/water debris?
13. Were there ample paper and soap supplies in all areas (rest room)
14. Did you enjoy the items you ordered? (put subjective answers in comment section).
15. At some point during your visit, did a staff member invite or encourage you to return?

8
Supplied by Paul Conforti.
 13

Exhibit 3

If the manager selected Desserts from the word cloud, the following verbatim responses would
be shown:

April 30, 2010 9:28 p.m. via iPhone: Special offers, featured "lighter" desserts for the summer
April 28, 2010 8:25 p.m. via BlackBerry: I would like to see more desserts on the menu, like
some from the bakery.
April 26, 2010 12:47 a.m. via Web: The personal and friendly service, the delicious drinks and
the sweet desserts!
April 21, 2010 10:16 p.m. via iPhone: Coupons for free desserts
April 17, 2010 12:48 p.m. via Web: I think that it would be better to focus on the ways that
dessert can be enjoyed. For example, small plates are a trend that has been popular for the better
part of 8 years now be them purist Tapas, or just conceptual Tapas, why not have small dessert
plate options? Fruit and Cheese Pairings? I think if Finale could offer an even wider range of
options, it would manage to lure in yet more people. Everyone wants to feel pampered and have a
treat, and if there are different sizes, and scopes to those treats, I think I'd be a regular. At present,
the menu breaks down between plated desserts, shared desserts, and the likes, but what if it
expanded an iced dessert section to allow for things like granitas, sorbets, and luxury frozen
items? I think if Finale perhaps would expand to dessert choices that would be more waistline
friendly, things that involve yogurts, fruits, sorbets, etc...perhaps I'd get in there more often.

April 10, 2010 9:17 p.m. via iPhone: More new desserts
April 9, 2010 11:58 p.m. via Mobile Web: More desserts

If the manager selected Prices from the word cloud, the following verbatim responses would be
shown:
April 30, 2010 8:50 p.m. via BlackBerry: Cheaper prices
April 29, 2010 11:27 p.m. via Web: Lower prices, better quality
April 28, 2010 10:35 p.m. via BlackBerry: Lower prices
April 28, 2010 8:26 p.m. via BlackBerry: Lower prices and better music
April 26, 2010 7:56 p.m. via Mobile Web: Lower prices.
April 23, 2010 4:29 p.m. via iPhone: Coupons, discounts, better dessert prices
April 17, 2010 12:21 p.m. via iPhone: Lower prices
April 16, 2010 11:04 p.m. via Mobile Web: Lower prices
April 15, 2010 9:24 p.m. via Mobile Web: Lower prices (or special deals like Harvard student
discount)
April 8, 2010 10:34 p.m. via Mobile Web: Lower prices
April 6, 2010 7:47 p.m. via Mobile Web: Lower prices
April 6, 2010 6:37 p.m. via Mobile Web: Lower prices
April 4, 2010 8:29 p.m. via Mobile Web: Somewhat lower prices
April 1, 2010 10:34 p.m. via Mobile Web: Lower prices
14