Being an early stage investor, I see a lot of MVPs. I mean a lot. The problem is that most people don’t build real MVPs, but descend into building a prototype, then beta, then an actual product and still call it an MVP. Remember an MVP is your most important Customer Development tool. It is an experiment designed to test your value proposition’s assumptions by measuring a behavior and learning from the results. MVPs should be used for a short while in order to validate your learning and then help you develop the plan for the actual product. After you have built a few MVPs and measured the results, what are the top three signs that you are done with your MVP and can start building your real product?
Sign #1 That You’re Done With Your MVP: You are getting diminishing returns on your learning
Typically when you start doing the Customer Development/MVP process, it is pretty brutal. Your ideas are shattered when they come into contact with real people. It can be long, hard, and painful at times. But when you stick with it you usually go through three stages of MVPing:
Shattered expectations
Hearing the same thing over and over
Diminishing returns
Usually after your first round of MVPs, you go back to the drawing board a little and make adjustments to your Business Model Canvas and test your new assumptions. Typically you get into a zone where everyone is saying the same thing: this is good. After a while you adjust the MVP some more based on that feedback and the feedback and behavior you are measuring is only giving you a very small incremental gain. This is when it is time to stop MVPing and build an actual product.
Sign #2 That You’re Done With Your MVP: You’ve been doing customer development for a really long time
I’ve met countless startups that were working on their MVPs for months or years. Typically a single MVP should last a few hours or a day. After that time, you process the data, apply the learnings, and then make another MVP that should only last a short time as well. In a perfect world, you would only be MVPing for a month or two at most since the results of all of your MVP experiments gave you enough data to build a product.
If you have been MVPing on the same idea for over six months, you need to reevaluate what are doing. Some startups clearly have a product (see #3) and some clearly do not. If you have not seen traction in a long time and have been grinding away on the same idea for several months, it is time to ask yourself some hard questions.
Sign #3 That You’re Done With Your MVP: You are actually making money
Sometimes I’ve seen startups that are still MVPing when they are actually making money on their product or service. Typically the startup doesn’t have a clear indication of what the pricing model should be or what the right customer segment is. Once you have enough “validating revenue” (typically around $250k ARR), stop MVPing and start A/B testing and using other tools (typically sales tricks) to figure out the best pricing and customer segmentation.
As you start your next venture, think about these three tips when you start your Customer Development so you do it effectively, but don’t do it too long. Good luck!
A lot of people misuse the term “MVP” or Minimum Viable Product. To be clear an MVP is not a beta, not a prototype, but rather an experiment designed to test your value proposition’s assumptions by measuring a behavior and learning from the results.
Back in the day, Dropbox did an MVP as just a video and Buffer was just a landing page. Both were experiments to determine if Dropbox or Buffer should even exist. Instead of guessing and building prototypes, they built the simplest of things in order to measure a user’s behavior. Today startups are building functional prototypes and calling them MVPs. They are better off building something they can learn from. Typically the first MVP doesn’t even have to be anything on a device or computer. For example, I once advised a new travel startup that wanted to give you one click access to a daily itinerary based on a map. They assumed that people wanted a map with pin points on it and times to follow. I told them to go to tourist spots and give people real maps with real pin points circled and an analog itinerary to follow. That was an MVP, it was an experiment (map) that measured (how many as a percent of total) a user behavior (did they use the map or not). Let’s take a look at how to build a better MVP.
Getting Started: Customer Segment and Value Proposition
The whole idea of an MVP is to measure an actual result against your expected result to prove or disprove your assumption. In order to do that you need data. The first place to start is to think about is your customer segment; you have to know who your target customers are going to be. Without knowing your exact segment (22–34 year old professional, urban women, single, living alone, earning over $75k), you won’t be able get the correct pool of users to test on.
After you define your customer segment, you define your value proposition. Too many people think that their value proposition is just the solution to the problem they are solving. That is incorrect: your value proposition is the delta between the current solution or workaround to the problem people are currently using and your solution. You measure your value proposition in terms of how much better your solution is compared to the solutions that exist today.
Let’s say you are solving a problem for buying movie tickets. Several solutions already exist; there are lots of web sites, apps, etc. Maybe your solution involves buying the tickets via SMS. Regardless, you have to think about what the alternatives to your solution are and compare them against that. One is simply buying the ticket at the box office. Here your alternative has value, but not tremendous value. Alternatively, let’s say you are developing a life saving cancer drug. The alternative without your solution could be death. In this case your solution would be incredibly valuable.
The Assumptions That Fuel Your Value Proposition
Underpinning your value proposition are your core assumptions. These are the things that would compel someone to buy your product or service. The job of the MVP is to test those underlying assumptions. The only way to successfully test those assumptions is by making a prediction of the result and comparing the behaviors that you measured up against your predictions. Your predictions should be based in fact, facts that would determine if you have a viable business or not. If you don’t make a prediction, then you will not have a way to determine success or failure of the MVP test.
Let’s say you are building a landing page, Buffer style. Your MVP will be to measure how many people give you their email address after your landing page described your product. You will have to drive traffic to your landing page, most likely by taking out some Facebook or Google AdWords ads. You want to measure the conversion rate of people who clicked on the ad (since you pay for click) to providing their email addresses. For example, if 100 people clicked on the ad and came to your page, but only 4 provided their email address, your conversion rate is 4%. (Not bad actually in e-commerce.)
Should 4% be your target? No. You need to determine your prediction based on facts and your business model. Let’s say you estimate spending $100 on Google AdWords to drive traffic to your MVP. If you have a conversion rate of 4%, it will then cost you $25 to acquire each customer. $25 is your CAC or customer acquisition cost. You need to estimate what your Customer Lifetime Value (CLV), or the amount of profit you expect to get out of each customer over the course of their relationship with you, is. At this stage it will be fairly inaccurate, but you need to ground your assumption in reality. (Future MVPs can test pricing.) Let’s say you make the CLV to be $21, based on a lot of factors in your business model. (I talk more about your CLV and CVC here.)
With a a CLV of $21 and a CAC of $25, you will lose $4 on each new customer you acquire. Or CLV ($21) — CAC ($25) = -$4.
For your MVP test, you will need a higher conversion rate/lower CAC rate in order to make a profit. For the first MVP test make a prediction that the conversion rate will be 5%, bringing your CAC down to $20. Or CLV ($21) — CAC ($20) = $1.
Interpreting The Results
Now with your assumptions based in some business reality, it is time to run the test. Typically the results are one of the three following numbers (remember you are aiming for 5% conversion):
0.021%
4.28%
17%
Let’s take 0.021%. This is an absolute failure, you can safely assume that your assumption is invalidated. Safest thing to do is declare the assumption invalid and go back to your value proposition and rethink it. If you have other assumptions associated with your value proposition, you can do some more MVP tests to determine if the entire value proposition is invalid or not. Chances are you will have to iterate your idea and value proposition some more.
What to do if you are at 4.28%? Technically it is invalid since you need 5% conversion rate in order to make any money. Should you just give up and go home? No. You should try some new UX and new design or different language and run the test again. Don’t run the test without changing anything! If your future tests with minor changes are at or over 5%, then you can declare your assumptions valid and move on to test the next one.
Let’s look at 17%. Woo-hoo, your assumptions are more than valid, you blew away your predictions. Verify that your test was fair and then declare your assumption valid and move on to test the next assumption.
Thats all there is to it! Only by clearly defining what success is and basing those numbers in a business reality is an MVP useful. Anything else is just a beta.
A lot of people misuse the term “MVP” or Minimum Viable Product. To be clear an MVP is not a beta, not a prototype, but rather an experiment designed to test your value proposition’s assumptions by measuring a behavior and learning from the results.
Back in the day, Dropbox did an MVP as just a video and Buffer was just a landing page. Both were experiments to determine if Dropbox or Buffer should even exist. Instead of guessing and building prototypes, they built the simplest of things in order to measure a user’s behavior. Today startups are building functional prototypes and calling them MVPs. They are better off building something they can learn from. Typically the first MVP doesn’t even have to be anything on a device or computer. For example, I once advised a new travel startup that wanted to give you one click access to a daily itinerary based on a map. They assumed that people wanted a map with pin points on it and times to follow. I told them to go to tourist spots and give people real maps with real pin points circled and an analog itinerary to follow. That was an MVP, it was an experiment (map) that measured (how many as a percent of total) a user behavior (did they use the map or not). Let’s take a look at how to build a better MVP.
Getting Started: Customer Segment and Value Proposition
The whole idea of an MVP is to measure an actual result against your expected result to prove or disprove your assumption. In order to do that you need data. The first place to start is to think about is your customer segment; you have to know who your target customers are going to be. Without knowing your exact segment (22-34 year old professional, urban women, single, living alone, earning over $75k), you won’t be able get the correct pool of users to test on.
After you define your customer segment, you define your value proposition. Too many people think that their value proposition is just the solution to the problem they are solving. That is incorrect: your value proposition is the delta between the current solution or workaround to the problem people are currently using and your solution. You measure your value proposition in terms of how much better your solution is compared to the solutions that exist today.
Let’s say you are solving a problem for buying movie tickets. Several solutions already exist; there are lots of web sites, apps, etc. Maybe your solution involves buying the tickets via SMS. Regardless, you have to think about what the alternatives to your solution are and compare them against that. One is simply buying the ticket at the box office. Here your alternative has value, but not tremendous value. Alternatively, let’s say you are developing a life saving cancer drug. The alternative without your solution could be death. In this case your solution would be incredibly valuable.
The Assumptions That Fuel Your Value Proposition
Underpinning your value proposition are your core assumptions. These are the things that would compel someone to buy your product or service. The job of the MVP is to test those underlying assumptions. The only way to successfully test those assumptions is by making a prediction of the result and comparing the behaviors that you measured up against your predictions. Your predictions should be based in fact, facts that would determine if you have a viable business or not. If you don’t make a prediction, then you will not have a way to determine success or failure of the MVP test.
Let’s say you are building a landing page, Buffer style. Your MVP will be to measure how many people give you their email address after your landing page described your product. You will have to drive traffic to your landing page, most likely by taking out some Facebook or Google AdWords ads. You want to measure the conversion rate of people who clicked on the ad (since you pay for click) to providing their email addresses. For example, if 100 people clicked on the ad and came to your page, but only 4 provided their email address, your conversion rate is 4%. (Not bad actually in e-commerce.)
Should 4% be your target? No. You need to determine your prediction based on facts and your business model. Let’s say you estimate spending $100 on Google AdWords to drive traffic to your MVP. If you have a conversion rate of 4%, it will then cost you $25 to acquire each customer. $25 is your CAC or customer acquisition cost. You need to estimate what your Customer Lifetime Value (CLV), or the amount of profit you expect to get out of each customer over the course of their relationship with you, is. At this stage it will be fairly inaccurate, but you need to ground your assumption in reality. (Future MVPs can test pricing.) Let’s say you make the CLV to be $21, based on a lot of factors in your business model. (I talk more about your CLV and CVC here.)
With a a CLV of $21 and a CAC of $25, you will lose $4 on each new customer you acquire. Or CLV ($21) – CAC ($25) = -$4.
For your MVP test, you will need a higher conversion rate/lower CAC rate in order to make a profit. For the first MVP test make a prediction that the conversion rate will be 5%, bringing your CAC down to $20. Or CLV ($21) – CAC ($20) = $1.
Interpreting The Results
Now with your assumptions based in some business reality, it is time to run the test. Typically the results are one of the three following numbers (remember you are aiming for 5% conversion):
0.021%
4.28%
17%
Let’s take 0.021%. This is an absolute failure, you can safely assume that your assumption is invalidated. Safest thing to do is declare the assumption invalid and go back to your value proposition and rethink it. If you have other assumptions associated with your value proposition, you can do some more MVP tests to determine if the entire value proposition is invalid or not. Chances are you will have to iterate your idea and value proposition some more.
What to do if you are at 4.28%? Technically it is invalid since you need 5% conversion rate in order to make any money. Should you just give up and go home? No. You should try some new UX and new design or different language and run the test again. Don’t run the test without changing anything! If your future tests with minor changes are at or over 5%, then you can declare your assumptions valid and move on to test the next one.
Let’s look at 17%. Woo-hoo, your assumptions are more than valid, you blew away your predictions. Verify that your test was fair and then declare your assumption valid and move on to test the next assumption.
Thats all there is to it! Only by clearly defining what success is and basing those numbers in a business reality is an MVP useful. Anything else is just a beta.
