Quibi was suppose to change the way you watch video on mobile phones.

The idea was to deliver HBO quality, 10 minute “quick bite” episodes for customers on the go. Think of it as a mash up of Netflix / TV / YouTube for smartphones.

The company raised over $1.8 billion in funding. Spent $1.4 billion and 2 years building the product. And, finally launched it in April 2020.

Only to find out that the customers did not care for it. And so, the service was shut down just 6 months later. 

What happened?

Quibi’s founders – Jeffrey Katzenberg and Meg Whitman – made the classic mistake of, 

Building the product without validating the idea first

They hoped that if they built it, the users would follow.  Unfortunately, in our digital world, that is not how things work.

In this case study, we will deep dive into:

1. Quibi’s story
2. Why they failed
3. What could they have done instead (hint: run quick experiments)
4. The business case for running experiments 

Lets get into each of them, shall we.

1. Quibi’s Story

On the surface, Quibi’s idea seemed rock solid

Quibi’s big idea was to provide HBO quality programming to customers on the go (commuting, waiting in line etc.).

The vision was two fold,

• Provide customers with a better choice. Instead of “blah” user generated content found on YouTube and TikTok. Quibi wanted customers to trade up their viewing habits by providing better quality TV.
• Deliver unique mobile viewing experience. Unlike Netflix or Hulu, the service would only produce mobile friendly shows, delivered in 10 minute increments (called “quick bites”).

The idea was such a big hit, that the company was able to raise $1.8 billion from some of the best named Hollywood studios and venture capitalist.

To make the idea a reality, Quibi decided to go all in

Given the positive feedback from investors, producers, and the press. The company decided to spend $1.4 billion upfront to build the best version of the product.

They believed that if they build it, the customers will follow.

But reality turned out to be quite different

Initial projections had the company adding over 7 million subscribers at the end of the first year. Unfortunately, Quibi could barely scrounge up 500,000 users in 6 months.

Given the dismal performance. The founders decided to shut down the service in Oct 2020.

What happened? 

The problem starts with a flawed understanding of risk.  

2. Why They Failed

Quibi’s success depended on the assumption that consumers would be willing to “trade up” their viewing habits

As you dig a little deeper into Quibi’s big idea. You realize that success was “inevitable”, only if the following assumption came true,

Customers want and will pay for Quick Bites (short form, HBO quality programming) on mobile screens. 

Get this wrong and everything else falls apart. It was the startup’s biggest risk.

You can see this assumption front and center in an interview by Axios in 2018. 

According to Meg (9:30), “People are watching a lot of videos on mobile phones. We want to trade them up, with better and or higher quality”. 

To address this risk, the founders decided to build the best product possible within thier budget

They believed that if they want to attract the most customers – to get them to “trade up” – they needed to build a pretty awesome product right off the bat.

They thought that if they built the best content, on the best platform, with all the necessary bells and whistles. The customers will automatically follow.

To that end, their product’s first version included,

1. Over 5000 episodes and 175 new shows.  Many of them from big names and Hollywood stars such as – Kevin Hart, Jennifer Lopez and Steven Spielberg – $1 billion
2. New mobile platform, complete with a ton of features.  Including the much touted (and very expensive to build) Turnstyle feature – $300 to $400 million
3. Online and offline marketing – $60 million 

All in all, the company ended up spending over $1.4 billion and 2 years on product development.

Unfortunately this approach of “if we build it they will come” never works

There are three key problems with this approach.  

First, it is the slowest, most expensive, and inefficient way to test an assumption.  Quibi spent $1.4 billion and 2 years to find out what their customers really thought. That is a lot of time and money to spend on an untested idea.  Especially when 9 out of 10 ideas fail.

Second, it reduces options.  By the time Quibi realized that they needed to pivot, they had run out of time, money, and patient investors.  Instead of learning from their launch and building the next (better) iteration.  The founders had no choice but to shut down.

Third, the failure is too visible. Failing small and fast is sexy. Failing big and slow can be humiliating. Just look at the bad press (here, here, and here) Quibi received. The media had a field day.

3. What They Could Have Done Instead

A better approach is to test your riskiest assumption first

Instead of building a the whole product, a less risky approach would have been to run tests on the riskiest assumption first.

And by testing, I don’t mean market research and / or focus group surveys.  When have these types of hypothetical studies ever delivered accurate insight and data?  

Instead, I am referring to the quick and cheap experiments Google, Facebook, Amazon and a plethora of startups routinely engage in.  Where they,

1. Build a cheap facsimile of the product (i.e prototypes)
2. Create a testing environment that mimics real life decisions as close as possible
3. Show it to users, gather feedback, and
4. Decide what to build / not to build based on data

If you are looking for examples, check out Zappos, Drop Box, Groupon, Gmail, AirBnB, and Buffer.

Here are two experiments that Quibi could have run instead

Experiment 1:  Launch a Quibi YouTube channel

To test the assumption if users want to see “quick bites”. Quibi could have produced up to 3 high quality shows, put them up on YouTube, and measured user’s engagement levels.

Why Run This Experiment:

Doing so would have allowed them to learn, 

1. If users like better quality episodes vs user generated content?
2. What type of users (demographics) would be attracted to “quick bites”?
3. How many liked and subscribed to the channel?  What were their viewing habits? How many viewed it on their mobile phones? How many kept coming back? How many engaged etc.
4. Does adding another series attract similar user views & data?  How / why does it change?

How They Could Have Set It Up:

Setting up this experiment would require,

1. Creating 3 high quality shows (10 episodes of 10 minutes each).
2. Posting shows on a Quibi YouTube channel. Simulate the Quibi experience by dropping one daily episode at a time 
3. Asking users to subscribe, share, promote, and tweet the channel on social media
4. If needed, market the episodes by buying YouTube ads targeting the preferred demographics

How Much It Would Have Cost:

As for the economics of the experiment, 

1. Time: 3 – 4 months to produce 3 shows + 2 months to monitor impact on YouTube
2. Cost: $1M to $500K per show + YouTube Channel (free) + $20K YouTube Ad spend (if needed)
3. Team Needed: Production Crew, Actors, Product Team, YouTube SME

Other Considerations:

As to, why leverage a platform like YouTube to run the experiment.  There are four key reasons.

First, YouTube is where the potential future customers are.  By leveraging the platform, Quibi can reach target users without incurring too much cost and time.

Second, the platform is also the competition.  If Quibi is going to win, it will need to compete against user generated content on YouTube (and TikTok, IG etc.).  Better to find out now how “quick bites” fare against the competition.

Third, YouTube comes ready with a ton of good data (watch time, retention, demographics, impressions etc.).  It’s a perfect data rich platform to run tests.

Fourth, the platform is free.  So there are no development costs.

Test Plan:

Here is what a potential test plan would look like.  (numbers used are hypothetical and for example purposes only).

You can download the test plan here.

Experiment 2:  Create a sales landing page (fake door test)

To determine if users are willing to buy the service, Quibi could have constructed a “fake door” test.  Where YouTube subscribers would be directed to a sales landing page and asked to sign up for the service.  

Keeping with the idea of being small, cheap, and fast.  The experiment could be as simple as…

The goal is to capture how many people would visit the landing page, click through the different price points, and part with their emails.

Why Run This Experiment:

By leveraging this approach, they would have learned,

1. How many users / subscribers would be willing to learn more about the service (i.e visit the landing page)?
2. Once presented with a price, which payment tier would they gravitate towards?
3. How many clicked on the “learn more button”?  Would they be willing to part with their email? (good indicator on willingness to buy)

How They Could Have Set It Up:

Compared to experiment 1, setting this up would be much easier.  It would require,

1. Creating a sales landing page, which would include service description + list of upcoming shows + price. You can learn more about landing pages here
2. Executing an online campaign to get users to visit the landing page (via a link in the YouTube videos and email campaigns to subscribers) 
3. Maintaining a database of subscribers (this is the first set of users to target when the service goes live) 
4. Recommend to run this in parallel with Experiment 1 for most insights

How Much It Would Have Cost:

As for the economics of the experiment, 

1. Time: 2 weeks to create landing page + 2 months to monitor impact on YouTube
2. Cost: $100K 
3. Team Needed: Web Developer, Product Team, Marketing SME (campaigns)

Test Plan:

Here is what a potential test plan would look like.  (numbers used are hypothetical and for example purposes only).

 You can download the test plan here.

4. The Business Case For Experimenting

The experiments would have provided Quibi with similar insights. Only much sooner and cost a whole lot less.

The business case for testing is straightforward.

The original approach took Quibi $1.4 billion and 2 years to find out that their customers did not want the service.  

By running the above experiments, it would have taken them $2 – $3 million and 4 months, to uncover the same truth.  

Doing so, would have not only saved them a ton of money (over $1 billion), time (18 months), and a whole lot of public shame.

Thus providing them with flexibility

While the financial benefit is quite clear. The more important benefit is flexibility.

By not spending so much time or money upfront, they would have had the option to either abandon the idea (no shame in that).

Or more likely pivot to building a better product that customers will love.