Measuring the effect of advertising has always been a significant challenge for marketers. The Super Bowl presents a particularly interesting opportunity to study individual ads that reach millions of consumers and represent a major investment for brands at $4 million plus production costs.
I will use two sources of data to look at this problem: Un-aided recall by a random sample of consumers and sales results achieved by Go Daddy after their investment in Super Bowl ads.
Super Bowl Ad Recall
Over the last few days we ran a survey to evaluate how well consumers recalled last year’s Super Bowl ads. We chose ten of the 2013 ads that consistently ranked among the top ads based on various sources. A random sample of people were presented with a screenshot and a descriptive title of each ad. The results are telling:
- On average, people could only remember 29% of all ads.
- Less than 20% of respondents could recall seven of the 12 ads.
- Aside from the four top brands, less than 15% of respondents could recall the remaining ads.
- On average, only 13.28% of people could associate the correct brand with an ad.
- For three of the 12 ads, not one person could to identify (or guess) the correct brands.
- Only two brands had a higher than a 12% brand association result.
- 44% of respondents could remember the Mercedes Benz ad, but only 11% could identify the ad with the correct brand. Even more alarming, while 50% could identify the RAM ad ‘God made a farmer’, only 5% could match the ad with the correct brand and 30% incorrectly associated the ad with a competing auto brand.
- Budweiser and Go Daddy came out as the winners with:
- 64% and 70% of people remembering their ads.
- 46% and 54% correctly associating the brands with their ads.
The first question we should ask: Is it worth spending millions of dollars for an ad when less than a third of viewers will remember ever seeing the ad, and less than 15% will correctly associate your brand with the ad?
Surely the study was done informally, with a relatively small sample, and has multiple bias: the way the questions were asked, the people who were exposed to the survey invitation and the profile of those who took the interest and spent the time in answering the survey.
It is also important to note that the survey only measures conscious brand recall. As we know, what people say, think, or remember does not always match how they behave.
2013 Super Bowl ads were not always effective in consciously increasing brand equity. The example of RAM trucks, where a third of respondents associated the brand with competitors, is very worrying. Even this year, some people on Twitter considered Rackspace (my current employer) as one of the top ads, while the company did not have one – but website building tool Squarespace did.
One conclusion could be that brands focus too much on entertainment value and not enough on ads that will compel customers to act, predispose viewers to buy the brand, or at least generate interest.
For example, this year’s RadioShack eighties-themed ad made most lists of viewers’ favorite ads. Yet as nostalgic as the ad was, it did not offer a reason for viewers to visit a store nor did it establish a point of differentiation for the struggling chain aside from trying to escape its association with the 80s. It will be interesting to see if the company survives. (update: it did not).
“It was kind of a waste of money, people are overly optimistic. Yes, it’s clever but people aren’t going to go out and buy anything because of it” said Michael Pachter, analyst at Wedbush.
At the time of writing, the company was trading at less than a tenth of its value as of April, 2010, with a market capitalization of $222 million and just announced they are shutting down.
Analysis of Go Daddy’s Super Bowl investments
Go Daddy has invested significantly in Super Bowl ads, often taking a controversial – or to be more precise, sexist – angle. Business Insider published a study last year on the ROI of that marketing investment that made a lot of sense: Let me summarize this point of view:
[box]Based on company-provided results, Go Daddy signed up 10,000 new customers after the Super Bowl. Dividing the cost of running the ad by the number of new customers, the cost per customer acquisition is $750 – that’s not even counting ad production costs. Considering Go Daddy’s main line of business is based on small transactions for yearly domain fees plus some up-sell to hosting services, the estimated average lifetime customer value of a customer is $55. These figures would imply the company lost almost $700 per customer.[/box]
Now, these numbers make sense. Anyone looking for a business case against running a Super Bowl ad will find a great story here. However, there is another point of view that, although is not as precise, is quite compelling.
Back in 2004, Go Daddy’s biggest problem was awareness. Most people who needed a domain had never heard of Go Daddy.
According to Barb Rechterman, CMO of Go Daddy, “Business looks pretty good. And that comes from our first Super Bowl ad. Right before that Super Bowl, our share of the domain registration market share was 16% of new domain names. Now, it’s about 50%.“
According to industry publication Domain Wire, the company’s revenue grew from $138 million in 2005 to $500 million in 2008 and $750 million in 2009. Can you attribute all of this growth to the Super Bowl ad investment? Not in absolute terms, but it certainly had a strong influence.
Conclusions
We can draw several conclusions, many of them already mentioned above:
- It is almost impossible to conduct a precise ROI study that shows causation instead of just correlation in any broad branding activity.
- Super Bowl ads focus too much on being clever and entertaining and not enough on having an impact on consumers: building a brand, compelling to action or even creating curiosity.
- The only goal for a Super Bowl ad, or for any advertising activity, is to generate revenue, directly or indirectly. Everything else is a waste of money. Most Super Bowl ad rankings are based on popularity, cuteness, cleverness or fun – not on effectiveness.
- Super Bowl ads are quickly forgotten and the sponsors of the entertainment in them ignored. However, there could be subconscious benefits that are harder to measure.
- While the ads did not seem to work for Go Daddy based on directly observed results, they seem to deserve a good chunk of the credit for the company’s meteoric growth in the last few years.
- The most effective ads are those with a very clear goal in mind. Go Daddy’s investment was designed to solve their biggest business problem: awareness. And in that, they succeeded.
- Execution matters: Go Daddy and Budweiser produced ads with significantly higher recall and association results a year after the ads ran.
For more on Super Bowl ads, I suggest the next blog be on the topic: 10 Marketing Observations from the 2014 Super Bowl ads which provides an analysis of the most interesting trends and specific commentary on ads.