Who Were the Super Bowl Losers? Not the 49ers

Super Bowl 2024 Advertiser Highlights

For the past three decades, my team members have been helping organizations understand why software fails under load and what to change to reduce the risk of failures in production. Events such as the Super Bowl provide examples where we can discuss technology patterns associated with software scalability risk. Nearly every modern organization has an event where revenue and information technology risk are joined. We would be happy to help you ensure that your Super Bowl event is not handicapped by a piece of software that does not meet the demands required. Feel free to reach out to me here on LinkedIn or directly at [email protected] .

How did it go?

In 2024, 57 national commercials aired during the Super Bowl, which doesn't include the local commercial inserts around the United States or in foreign markets. In total, six commercial campaigns "failed," which equates to a little over ten percent (Figure 1). This ratio aligns well with the previous year's failures. That the percentage was not higher does speak well about the majority of the advertising campaigns and their readiness. A complete list of advertisers monitored during the event that did not experience failures appears at the very end of this document as a scorecard.

The remainder of the failures were ancillary events --items related to the Super Bowl but didn't have a commercial campaign. Some of them were expected, such as takeout pizza. Others were a complete surprise due to events happening live during the Super Bowl.

The most significant story of this year's game was the failure of streaming services. Paramount Plus, Hulu, TSN Sports, Verizon, and Tidal's audio streaming services all had issues. Neilsen reported that as many as 123.4 Million people watched the Super Bowl on its platform -- a little over a quarter from streaming users.

Poor performance is a very personal thing. We all know when we are not receiving it. Second, it impacts relationships with providers of poor performance. Matt Heusser , managing director of Excelon Development, spoke about his experience with the Super Bowl streaming at his house. Matt is a quality assurance professional and co-author of Software Testing Strategies: A testing guide for the 2020s with Michael Larsen. Matt indicated the audio was out of sync with the video, ranging from seven to twenty seconds off. He could hear notifications of touch-down events while plays were in progress. It was even worse during commercials, with the video of one commercial on screen and the audio for a second one playing.

Very quickly, the game became unwatchable with such a large disjoint between the audio and video tracking. Before Matt finished the interview, he indicated he was planning to terminate his Paramount+ on the same day if possible. His experience was that bad.

Looking at the tweets from last night, Matt was not the only one disturbed by the performance of Paramount+. I suspect that loss is far more than one as a result. Paramount, already struggling before its Super Bowl streaming challenges, laid off 800 people, or 3% of its workforce, on Monday after the Super Bowl.

Poor software performance has financial consequences. Marketing professionals tell us that people who encounter a down or poor-performing website and who depart are unlikely to return. How unlikely? Greater than four out of five (88 percent) will seek a different service opportunity. Only when other market options are exhausted will a consumer return to a poor-performing site to see if performance has improved. This abandoning user represents lost (initial) revenue and future income (opportunity costs).

Other streaming services also experienced issues. Various tweets on X, formerly known as Twitter, reported that Hulu had several service interruptions. Significant complaints followed Verizon and its streaming of the halftime show. In Canada, TSN Sports also had streaming interruptions. And finally, Tidal, an audio streaming service, had an outage related to Beyonce announcing two new singles and a new album. Beyonce was the source of three outages: the halftime show (with Verizon), Tidal (looking for her latest singles to download), and her site, Beyonce.com, where people went for information on her upcoming album and tour. The halftime show with Beyonce and Usher averaged an even higher number of viewers than the Super Bowl itself, at a combined 129 Million viewers.

X is an excellent source of data related to site outages under load. You can find daily examples associated with tickets, new product drops, test scores, targets of news stories, and (yes) even outages related to the Super Bowl and its commercials.

Taking a quick technical look at the streaming services and what may have happened, Paramount+ is historically an on-demand service. Such on-demand services customarily leverage a technology called unicast for streaming. A new connection is set up for each user that requires maintenance from the system to send encoded streaming data.

The larger the number of users, the higher the overhead in maintaining the connections, encoding, and sending. Overloaded encoders can easily cause the audio and video tracks to be significantly out of sync.

Live broadcasts demand a different model similar to traditional signal transmission for terrestrial radio and television -- multicast. With multicast, a single stream is encoded and sent to a specific address. This is similar to the broadcast channel on your television. A multicast client then 'subscribes' to the channel, receiving the broadcasted data stream for playback locally. The encoder overhead is low. The session overhead is low.

For Tidal, Beyonce's popularity overwhelmed the system with requests. There is some irony in that her husband, Jay-Z, founded Tidal and sits on the board of directors for the company that acquired the system. Of all sites, Tidal should have had the inside track on what was about to be announced. It failed, just as other sites did, from the extraordinary request load generated by the Super Bowl.

Tidal may have an edge that the video streaming services do not have - exclusive content. As one of only a handful of providers of Beyonce's highest-quality digital streams, her fans will likely return when the site is operational. Not all sites are so lucky.

Two additional industries help us understand the financial consequences of poor software performance, and both are aligned to the Super Bowl in 2024. The first is the delivery pizza industry, and the second is sports betting.

One week ago, Domino's Pizza noted in a Fox Business article that the Super Bowl was one of its business days of the year.

In a Monday press release, Domino's explained that the Super Bowl was one of its five busiest delivery days of the year. "Domino's typically sells over two million pizzas on game day – about 44% more than on a normal Sunday," the pizza chain said. "The most popular game day pizza topping is pepperoni."

Two of Domino's market competitors, Pizza Hut and Papa John's Pizza, had angry posts on X from users who were unhappy about their website performance. It makes sense that where the Super Bowl is one of Domino's largest days of the year, it is likely true for its competition. The American Pizza Community website notes that in 2023, 12.5 million pizzas were sold for game day.

When Pizza Hut and Papa Johns have issues on their website, where will you turn for pizza? Typically, a close substitute is another pizza provider. What one pizza provider loses on game day, another gains. There is a loss of revenue for one and a boost for another -- all related to poor website ordering performance. Given human nature, there is a risk that the person affected will call the new pizza provider for a while afterward, or they may end up with a pizza they have a taste preference for and now be lost as a revenue source. This consequence is related to poor software performance. A website is slow or crashes, and revenue is lost.

That brings us to sports betting.

According to the X community, Fanduel and Boyle Sports had issues before the game kickoff. Fanduel was also a Super Bowl commercial advertiser. There is a narrow window before a game to make your bets. If one avenue is closed and someone is insistent on making a bet, the competition will get the revenue. Imagine you are Fanduel -- not only are you picking up the costs for a multi-million-dollar advertisement on the Super Bowl, but you are also losing revenue every time someone experiences a performance issue when trying to make a bet.

Poor software performance impacts revenue through the loss of an initial sale, orphaning of a customer revenue stream, or opportunity costs associated with the loss of future sales. The Super Bowl is a very in-your-face event that highlights these issues year after year. In addition to the streaming, takeout, and sports betting industries, other failures surrounded the event. See the attached graphic for additional information.

There were two well-known patterns in play that likely impacted site outages:

1. The Pattern of "Free." Both HEB Grocers and Intuit Turbotax were giving away items of value to people who visited their site. For HEB this involved groceries for life and with Intuit a sweepstakes contest with a tease of one million USD.

2. The celebrity effect. The popularity of Beyonce drove additional volume during the halftime show, load against Tidal for her two new singles, and load against her personal website.

How can these world-class brands prevent future high-volume, high-risk ad failures? Simplifying the technical components used always has a benefit. Reduce the number of features and files associated with a landing page, removing (or minimizing) dynamic elements, and coordinating with partners (in the case of Tidal) to plan for the anticipated loads from the event.

Here are some strategies marketing and ad executives should consider before going live:

• Have a completely static landing page that is served completely from your Content Delivery Network provider

• Know your audience. For an event such as the Super Bowl, optimize it for devices most commonly used to access your site. For an event such as the Super Bowl, this is a mobile device. This leads to design considerations for the number and sizes of assets being delivered to the device.

• Run your production infrastructure on a separate set of servers from your marketing website front end. You want your production infrastructure to be minimally affected by an outage on the marketing web infrastructure.

Ultimately, if in-house performance engineering technical resources are non-existent or over bandwidth, engaging a highly reputable performance engineering consultant with experience in spot ads should be a top priority. Contracting with them during campaign design is best to enable proactive martech stack selection, integrations, creative development and capacity discussions, implementation, and testing – before the ad goes live.

My team at QA Consultants stands ready to assist organizations in understanding the technical risk associated with scalability and determining what items to address related to architecture, configuration, and code to reduce your application's risk.

Super Bowl LVIII Commercial Outages

Every Super Bowl experiences a certain number of failures. From drops to non-responsive websites and landing pages to application outages, each year multiple brands underestimate the interest (volume) its ad will generate, neglects to perform accurate load testing, and/or chooses the wrong tech stack to accommodate the campaign goals and objectives. Super Bowl LVIII was no different. Following are the failures that dominated the event this year.

Super Bowl LVIII Score Card

✅︎ Domino's

❌ Pizza Hut

❌ Papa Johns

✅︎ Little Caesars Pizza

✅︎ Marco's Pizza (Marco's Franchising, LLC)

✅︎ Buffalo Wild Wings

✅︎ Marswrigleytreats

✅︎ Pringles

✅︎ MountainDew

✅︎ BetMGM

✅︎ Dove

✅︎ popeyes

✅︎ PepsiCo

✅︎ BMW Manufacturing Co., LLC

✅︎ Kawasaki Motors Corp., U.S.A.

✅︎ Unilever

❌ HelloFresh

❌ H-E-B

✅︎ Mondelēz International

✅︎ Booking.com

❌ Intuit TurboTax

✅︎ Google

❌ Verizon

✅︎ Uber

✅︎ National Football League (NFL)

✅︎ Nestlé

✅︎ Frito-Lay

✅︎ Volkswagen

✅︎ Skechers

✅︎ Toyota Motor Corporation

✅︎ T-Mobile

✅︎ ELF Cosméticos Pro

✅︎ Statefarm Insurance

✅︎ Dunkin'

✅︎ NYX Professional Makeup

✅︎ DoorDash

✅︎ Square

✅︎ E*TRADE from Morgan Stanley

✅︎ Danone

✅︎ Anheuser-Busch

✅︎ Budweiser Co

❌ BoyleSports

❌ FanDuel

✅︎ Foundation to Combat Antisemitism

✅︎ Kia Worldwide

✅︎ The Hershey Company

✅︎ Pluto TV

✅︎ Procter & Gamble

✅︎ Etsy

✅︎ BIC

✅︎ Molson Coors Beverage Company

live X feed data source