How to Validate Your Tech Startup Idea
How to Validate Your Tech Startup Idea
How to Validate Your Tech Startup Idea
December 12, 2020
If you’re thinking about the next big idea that matches or even beats Airbnb, Netflix, and Uber, then you’re on the right track.
Entrepreneurs have to keep generating ideas because you never know which one will stick.
But the founding stories of these successful startups were oversimplified and glorified. Most of us would never hear about how many ideas and failed validations these founders had to go through in landing this one business model.
What I want to talk about here: validation.
In the last couple of years, mainly because of the Lean Startup movement, a lot of us founders are obsessed with talking to users and finding ways to quickly validate ideas.
So, why I wrote this article
Because I’m a big fan of talking to people for validation, but the world of founders often jumps to this “user interview” step immediately.
I found that there is a big missing step before founders actually talk to people. Many of them miss it, and some of them ignore it.
When we founders go out to talk to users to discover whether they have “the problem” we’re thinking of, we usually already have a rough solution in mind. A lot of wise entrepreneurs advise us not to, but trust me, we still do because we love coming up with solutions as humans.
Since we already have this “solution” in mind, we should be evaluating whether this idea is worth pursuing before all the hard work of finding and talking to people.
Why is this important? Because not every problem is worth solving.
This was a tough lesson for me to share because it was one of those moments I couldn’t believe I actually made such a rookie mistake. So I want this to be a guiding principle for me and other founders to evaluate new startup ideas.
The Story of Toasty
I ran a startup called Toasty for 18 months in 2019-2020. The whole startup was built upon my vision of how we could improve human interactions with one another.
In the journey, we pivoted once because we knew our idea and direction were bad, then we pivoted again because of COVID-19.
And this post is about the first 6 months of Toasty, before our first pivot.
We built a matchmaking platform for conference-goers to screen and set up meetings with each other before the actual date. The goal was to enhance their networking experience so that it was worthwhile to spend days at the conference, especially when tickets were so expensive or if attendees traveled overseas.
Here is us traveling to Shanghai for the first conference that used Toasty! We wanted to be there to observe and talk to end-users.
The web app was beautifully designed to allow any attendee to see who else was going to the conference and set up meetings beforehand.
How did we come up with this idea?
Honestly, we had zero ideas about “the problem” and “the solution”.
We ran about a dozen user interviews in the conference and exhibition space and we could see a trend emerging. Most organizers were searching for new technologies that could level up the attendee’s experience. A good percentage of them had started using different products in the market but were not so happy about them and the outcomes.
We researched the space and the competitors and confirmed that there weren’t many competitors in the space, but for the few existing players, we thought they were doing fairly well, built a solid brand, and had grown their team to about 30 people in a few years.
With these data points, we jumped in without thinking too much.
6 months later, after working with conferences around Asia, I made the decision to pivot our entire direction to work on a new product. I didn’t see how this matchmaking platform could be sustainable to be a solid venture.
It was one of the toughest calls I had to make.
Toasty analyzed everyone's profile and recommended people you should meet.
What we learned about this business model
Before the decision to pivot, I took some time to consolidate the learnings, and here is a summary.
Sales cycle is long
November is too early to talk about a conference in March next year.
Professional conference organizers typically host 1 to 10 conferences a year, depending on the scale of each one. Most of them don’t think about the operations or experience until the last few months. They kicked off marketing pretty early though. This means the sales cycle is long (2-3 months) for a contract size that is not that big (low four digits USD).
Frequency of usage is super low
Since conferences don’t happen very often, the need to use a matchmaking platform is super low, from once a year to a few times a year.
Buyers, operators, and end-users are three different groups of people
The marketing/experience department searches and compares different software options. The operations department handles the logistics of the business matchmaking. And finally, the conference attendees use the software. When the buyer and user aren’t the same person, the chance of having a failed product experience skyrocketed.
Zero viral effect
The end users (attendees) are exposed to our matchmaking platform once (during the conference week), and after the conference, our product is completely out of everyone’s mind. The likelihood for someone to talk and share with another person (organizers or attendees) is very low.
One-off payment term
A one-off payment is also hard to collect. Here they just never paid us 50% of the contract.
Since it is not a SaaS subscription platform but a B2B platform, organizers pay a deposit and then the remaining payment after the conference is done. Then for the next conference in months or a year, it is likely that another person would be in charge of running the show. Pitch once, sell once. Not scalable.
Market size is small
We made a naive assumption that because it is a software platform, it would be okay to have a small local market. However, the matchmaking platform actually comes with a lot of other parts, such as on-site logistics, promotion, table arrangement, technical support, etc. Even if we could work this out and serve all the conferences in Asia, this is just not a big market, and it is not growing fast at all.
Matchmaking is not top of the organizer’s mind
Keynote and Sponsors make the conference, the other things are all icing on the cake.
Yes, they want it and they look for it. But when we dug deeper into the hierarchy of priorities from the organizer’s point of view, these other things are top of their mind: ticket sales, sponsors, booths, speakers, or even the coffee corner. An enhancement of experience using technology? “I’ll think about it when I have time.”
Willingness to pay
Even though they want it, the time constraint of hosting a conference and matchmaking being a low priority, they’re always looking for the cheaper solution out there to fill the need and save money.
Non-existent feedback loop
The key to growing a startup product is to collect a ton of feedback to improve iteratively. Most of the buyers are not on-site at the matchmaking site, so they cannot offer any feedback. And attendees only get to use our platform for 1-2 days, so it is extremely difficult to follow up to get their feedback as well.
In hindsight, this was a terrible idea
Waving goodbye to our first product.
I don’t think this is an impossible idea since a few companies have built a pretty successful business around this niche. It can be a business, just not one that could grow fast and delight a lot of people. It could work in 2015, not 2019.
The world has changed a lot, now we wouldn’t want to work on this type of idea because the cost is high, growth is slow, and product iteration is extremely lacking.
When competition in each segment is fierce, we must pick ideas that have a better chance of success.
So reflecting back, I came up with this list to help me evaluate future ideas.
How I’d evaluate my future startup ideas (especially for B2B SaaS)
1. Is my idea part of a global trend?
It is much better to ride with the wave than to go against it. Zapier played right in the global trend of booming Marketing Technology when it started in 2011. What did they do? They connected all these platforms.
Source: SaaS Marketer
2. Is the product easy to use? Can users self-onboard?
This makes sure the rate of growth can be exponential. A product that raises the least confusing questions in the user’s mind wins.
3. Is it easy for people to tell their colleagues or friends? Can they try it out immediately?
The moment right after “word of mouth”, when someone just hears about your product from another person, is the best time to capture interest. They should be able to try it out right away.
4. How long is the time to value or AHA moment? Can users use it in seconds, minutes, or days and weeks?
The shorter the time to value, the more likely for users to fall in love with the product.
5. What is the frequency of use? Is it daily, weekly, monthly, or annually?
The more frequently they use the product, the more likely a habit will be formed, leading to higher stickiness.
6. Who are the players right now? How big are they?
This is a tricky one. Without competitors, I’d question the demand for this space. With fast-growing competitors, I’d question the disadvantage. With a number of companies around 20-30 employees, I’d question the growth potential of the space.
7. Can I build a 10x product in this space? Do I and the co-founders have the background and expertise to create a significantly better product than everyone else?
If not 10x better, then there is no point in working on this idea because it is not convincing enough to acquire users. A 10% better product will have a super hard time asking users to form new habits.
8. How easy is it to get to the target audience? How big is the market?
E.g. restaurant is a terrible audience to go after. (See here) If it requires salespeople to go door-to-door to get to the audience, the idea won’t fly. Also, after focusing on a niche audience, it must be possible to expand to other groups.
Over to you
As a founder, it is your role to evaluate an idea and the relevant hypotheses before wasting resources on building it or even just researching it.
Assume -> Validate -> Build -> Feedback. Repeat.
And now with the number of #NoCode products out there, it is so easy for founders to run validations quickly.
Of course, after the hard lesson here, we pivoted to a product that was part of the global trend, easy to use, fast to spread, and quick to value. Toasty is now a video meeting platform focused on collaboration and engagement and is getting a lot more users than the matchmaking platform. As a side note, I stepped down from my CEO role in Oct 2020 and Eric Tsang is now leading the team.
To wrap this up, I’ve put these 8 questions onto this Google Sheet so you can clone and answer them honestly about your idea and see whether you still want to pursue it afterward. Click here to download it.
If you’re thinking about the next big idea that matches or even beats Airbnb, Netflix, and Uber, then you’re on the right track.
Entrepreneurs have to keep generating ideas because you never know which one will stick.
But the founding stories of these successful startups were oversimplified and glorified. Most of us would never hear about how many ideas and failed validations these founders had to go through in landing this one business model.
What I want to talk about here: validation.
In the last couple of years, mainly because of the Lean Startup movement, a lot of us founders are obsessed with talking to users and finding ways to quickly validate ideas.
So, why I wrote this article
Because I’m a big fan of talking to people for validation, but the world of founders often jumps to this “user interview” step immediately.
I found that there is a big missing step before founders actually talk to people. Many of them miss it, and some of them ignore it.
When we founders go out to talk to users to discover whether they have “the problem” we’re thinking of, we usually already have a rough solution in mind. A lot of wise entrepreneurs advise us not to, but trust me, we still do because we love coming up with solutions as humans.
Since we already have this “solution” in mind, we should be evaluating whether this idea is worth pursuing before all the hard work of finding and talking to people.
Why is this important? Because not every problem is worth solving.
This was a tough lesson for me to share because it was one of those moments I couldn’t believe I actually made such a rookie mistake. So I want this to be a guiding principle for me and other founders to evaluate new startup ideas.
The Story of Toasty
I ran a startup called Toasty for 18 months in 2019-2020. The whole startup was built upon my vision of how we could improve human interactions with one another.
In the journey, we pivoted once because we knew our idea and direction were bad, then we pivoted again because of COVID-19.
And this post is about the first 6 months of Toasty, before our first pivot.
We built a matchmaking platform for conference-goers to screen and set up meetings with each other before the actual date. The goal was to enhance their networking experience so that it was worthwhile to spend days at the conference, especially when tickets were so expensive or if attendees traveled overseas.
Here is us traveling to Shanghai for the first conference that used Toasty! We wanted to be there to observe and talk to end-users.
The web app was beautifully designed to allow any attendee to see who else was going to the conference and set up meetings beforehand.
How did we come up with this idea?
Honestly, we had zero ideas about “the problem” and “the solution”.
We ran about a dozen user interviews in the conference and exhibition space and we could see a trend emerging. Most organizers were searching for new technologies that could level up the attendee’s experience. A good percentage of them had started using different products in the market but were not so happy about them and the outcomes.
We researched the space and the competitors and confirmed that there weren’t many competitors in the space, but for the few existing players, we thought they were doing fairly well, built a solid brand, and had grown their team to about 30 people in a few years.
With these data points, we jumped in without thinking too much.
6 months later, after working with conferences around Asia, I made the decision to pivot our entire direction to work on a new product. I didn’t see how this matchmaking platform could be sustainable to be a solid venture.
It was one of the toughest calls I had to make.
Toasty analyzed everyone's profile and recommended people you should meet.
What we learned about this business model
Before the decision to pivot, I took some time to consolidate the learnings, and here is a summary.
Sales cycle is long
November is too early to talk about a conference in March next year.
Professional conference organizers typically host 1 to 10 conferences a year, depending on the scale of each one. Most of them don’t think about the operations or experience until the last few months. They kicked off marketing pretty early though. This means the sales cycle is long (2-3 months) for a contract size that is not that big (low four digits USD).
Frequency of usage is super low
Since conferences don’t happen very often, the need to use a matchmaking platform is super low, from once a year to a few times a year.
Buyers, operators, and end-users are three different groups of people
The marketing/experience department searches and compares different software options. The operations department handles the logistics of the business matchmaking. And finally, the conference attendees use the software. When the buyer and user aren’t the same person, the chance of having a failed product experience skyrocketed.
Zero viral effect
The end users (attendees) are exposed to our matchmaking platform once (during the conference week), and after the conference, our product is completely out of everyone’s mind. The likelihood for someone to talk and share with another person (organizers or attendees) is very low.
One-off payment term
A one-off payment is also hard to collect. Here they just never paid us 50% of the contract.
Since it is not a SaaS subscription platform but a B2B platform, organizers pay a deposit and then the remaining payment after the conference is done. Then for the next conference in months or a year, it is likely that another person would be in charge of running the show. Pitch once, sell once. Not scalable.
Market size is small
We made a naive assumption that because it is a software platform, it would be okay to have a small local market. However, the matchmaking platform actually comes with a lot of other parts, such as on-site logistics, promotion, table arrangement, technical support, etc. Even if we could work this out and serve all the conferences in Asia, this is just not a big market, and it is not growing fast at all.
Matchmaking is not top of the organizer’s mind
Keynote and Sponsors make the conference, the other things are all icing on the cake.
Yes, they want it and they look for it. But when we dug deeper into the hierarchy of priorities from the organizer’s point of view, these other things are top of their mind: ticket sales, sponsors, booths, speakers, or even the coffee corner. An enhancement of experience using technology? “I’ll think about it when I have time.”
Willingness to pay
Even though they want it, the time constraint of hosting a conference and matchmaking being a low priority, they’re always looking for the cheaper solution out there to fill the need and save money.
Non-existent feedback loop
The key to growing a startup product is to collect a ton of feedback to improve iteratively. Most of the buyers are not on-site at the matchmaking site, so they cannot offer any feedback. And attendees only get to use our platform for 1-2 days, so it is extremely difficult to follow up to get their feedback as well.
In hindsight, this was a terrible idea
Waving goodbye to our first product.
I don’t think this is an impossible idea since a few companies have built a pretty successful business around this niche. It can be a business, just not one that could grow fast and delight a lot of people. It could work in 2015, not 2019.
The world has changed a lot, now we wouldn’t want to work on this type of idea because the cost is high, growth is slow, and product iteration is extremely lacking.
When competition in each segment is fierce, we must pick ideas that have a better chance of success.
So reflecting back, I came up with this list to help me evaluate future ideas.
How I’d evaluate my future startup ideas (especially for B2B SaaS)
1. Is my idea part of a global trend?
It is much better to ride with the wave than to go against it. Zapier played right in the global trend of booming Marketing Technology when it started in 2011. What did they do? They connected all these platforms.
Source: SaaS Marketer
2. Is the product easy to use? Can users self-onboard?
This makes sure the rate of growth can be exponential. A product that raises the least confusing questions in the user’s mind wins.
3. Is it easy for people to tell their colleagues or friends? Can they try it out immediately?
The moment right after “word of mouth”, when someone just hears about your product from another person, is the best time to capture interest. They should be able to try it out right away.
4. How long is the time to value or AHA moment? Can users use it in seconds, minutes, or days and weeks?
The shorter the time to value, the more likely for users to fall in love with the product.
5. What is the frequency of use? Is it daily, weekly, monthly, or annually?
The more frequently they use the product, the more likely a habit will be formed, leading to higher stickiness.
6. Who are the players right now? How big are they?
This is a tricky one. Without competitors, I’d question the demand for this space. With fast-growing competitors, I’d question the disadvantage. With a number of companies around 20-30 employees, I’d question the growth potential of the space.
7. Can I build a 10x product in this space? Do I and the co-founders have the background and expertise to create a significantly better product than everyone else?
If not 10x better, then there is no point in working on this idea because it is not convincing enough to acquire users. A 10% better product will have a super hard time asking users to form new habits.
8. How easy is it to get to the target audience? How big is the market?
E.g. restaurant is a terrible audience to go after. (See here) If it requires salespeople to go door-to-door to get to the audience, the idea won’t fly. Also, after focusing on a niche audience, it must be possible to expand to other groups.
Over to you
As a founder, it is your role to evaluate an idea and the relevant hypotheses before wasting resources on building it or even just researching it.
Assume -> Validate -> Build -> Feedback. Repeat.
And now with the number of #NoCode products out there, it is so easy for founders to run validations quickly.
Of course, after the hard lesson here, we pivoted to a product that was part of the global trend, easy to use, fast to spread, and quick to value. Toasty is now a video meeting platform focused on collaboration and engagement and is getting a lot more users than the matchmaking platform. As a side note, I stepped down from my CEO role in Oct 2020 and Eric Tsang is now leading the team.
To wrap this up, I’ve put these 8 questions onto this Google Sheet so you can clone and answer them honestly about your idea and see whether you still want to pursue it afterward. Click here to download it.
If you’re thinking about the next big idea that matches or even beats Airbnb, Netflix, and Uber, then you’re on the right track.
Entrepreneurs have to keep generating ideas because you never know which one will stick.
But the founding stories of these successful startups were oversimplified and glorified. Most of us would never hear about how many ideas and failed validations these founders had to go through in landing this one business model.
What I want to talk about here: validation.
In the last couple of years, mainly because of the Lean Startup movement, a lot of us founders are obsessed with talking to users and finding ways to quickly validate ideas.
So, why I wrote this article
Because I’m a big fan of talking to people for validation, but the world of founders often jumps to this “user interview” step immediately.
I found that there is a big missing step before founders actually talk to people. Many of them miss it, and some of them ignore it.
When we founders go out to talk to users to discover whether they have “the problem” we’re thinking of, we usually already have a rough solution in mind. A lot of wise entrepreneurs advise us not to, but trust me, we still do because we love coming up with solutions as humans.
Since we already have this “solution” in mind, we should be evaluating whether this idea is worth pursuing before all the hard work of finding and talking to people.
Why is this important? Because not every problem is worth solving.
This was a tough lesson for me to share because it was one of those moments I couldn’t believe I actually made such a rookie mistake. So I want this to be a guiding principle for me and other founders to evaluate new startup ideas.
The Story of Toasty
I ran a startup called Toasty for 18 months in 2019-2020. The whole startup was built upon my vision of how we could improve human interactions with one another.
In the journey, we pivoted once because we knew our idea and direction were bad, then we pivoted again because of COVID-19.
And this post is about the first 6 months of Toasty, before our first pivot.
We built a matchmaking platform for conference-goers to screen and set up meetings with each other before the actual date. The goal was to enhance their networking experience so that it was worthwhile to spend days at the conference, especially when tickets were so expensive or if attendees traveled overseas.
Here is us traveling to Shanghai for the first conference that used Toasty! We wanted to be there to observe and talk to end-users.
The web app was beautifully designed to allow any attendee to see who else was going to the conference and set up meetings beforehand.
How did we come up with this idea?
Honestly, we had zero ideas about “the problem” and “the solution”.
We ran about a dozen user interviews in the conference and exhibition space and we could see a trend emerging. Most organizers were searching for new technologies that could level up the attendee’s experience. A good percentage of them had started using different products in the market but were not so happy about them and the outcomes.
We researched the space and the competitors and confirmed that there weren’t many competitors in the space, but for the few existing players, we thought they were doing fairly well, built a solid brand, and had grown their team to about 30 people in a few years.
With these data points, we jumped in without thinking too much.
6 months later, after working with conferences around Asia, I made the decision to pivot our entire direction to work on a new product. I didn’t see how this matchmaking platform could be sustainable to be a solid venture.
It was one of the toughest calls I had to make.
Toasty analyzed everyone's profile and recommended people you should meet.
What we learned about this business model
Before the decision to pivot, I took some time to consolidate the learnings, and here is a summary.
Sales cycle is long
November is too early to talk about a conference in March next year.
Professional conference organizers typically host 1 to 10 conferences a year, depending on the scale of each one. Most of them don’t think about the operations or experience until the last few months. They kicked off marketing pretty early though. This means the sales cycle is long (2-3 months) for a contract size that is not that big (low four digits USD).
Frequency of usage is super low
Since conferences don’t happen very often, the need to use a matchmaking platform is super low, from once a year to a few times a year.
Buyers, operators, and end-users are three different groups of people
The marketing/experience department searches and compares different software options. The operations department handles the logistics of the business matchmaking. And finally, the conference attendees use the software. When the buyer and user aren’t the same person, the chance of having a failed product experience skyrocketed.
Zero viral effect
The end users (attendees) are exposed to our matchmaking platform once (during the conference week), and after the conference, our product is completely out of everyone’s mind. The likelihood for someone to talk and share with another person (organizers or attendees) is very low.
One-off payment term
A one-off payment is also hard to collect. Here they just never paid us 50% of the contract.
Since it is not a SaaS subscription platform but a B2B platform, organizers pay a deposit and then the remaining payment after the conference is done. Then for the next conference in months or a year, it is likely that another person would be in charge of running the show. Pitch once, sell once. Not scalable.
Market size is small
We made a naive assumption that because it is a software platform, it would be okay to have a small local market. However, the matchmaking platform actually comes with a lot of other parts, such as on-site logistics, promotion, table arrangement, technical support, etc. Even if we could work this out and serve all the conferences in Asia, this is just not a big market, and it is not growing fast at all.
Matchmaking is not top of the organizer’s mind
Keynote and Sponsors make the conference, the other things are all icing on the cake.
Yes, they want it and they look for it. But when we dug deeper into the hierarchy of priorities from the organizer’s point of view, these other things are top of their mind: ticket sales, sponsors, booths, speakers, or even the coffee corner. An enhancement of experience using technology? “I’ll think about it when I have time.”
Willingness to pay
Even though they want it, the time constraint of hosting a conference and matchmaking being a low priority, they’re always looking for the cheaper solution out there to fill the need and save money.
Non-existent feedback loop
The key to growing a startup product is to collect a ton of feedback to improve iteratively. Most of the buyers are not on-site at the matchmaking site, so they cannot offer any feedback. And attendees only get to use our platform for 1-2 days, so it is extremely difficult to follow up to get their feedback as well.
In hindsight, this was a terrible idea
Waving goodbye to our first product.
I don’t think this is an impossible idea since a few companies have built a pretty successful business around this niche. It can be a business, just not one that could grow fast and delight a lot of people. It could work in 2015, not 2019.
The world has changed a lot, now we wouldn’t want to work on this type of idea because the cost is high, growth is slow, and product iteration is extremely lacking.
When competition in each segment is fierce, we must pick ideas that have a better chance of success.
So reflecting back, I came up with this list to help me evaluate future ideas.
How I’d evaluate my future startup ideas (especially for B2B SaaS)
1. Is my idea part of a global trend?
It is much better to ride with the wave than to go against it. Zapier played right in the global trend of booming Marketing Technology when it started in 2011. What did they do? They connected all these platforms.
Source: SaaS Marketer
2. Is the product easy to use? Can users self-onboard?
This makes sure the rate of growth can be exponential. A product that raises the least confusing questions in the user’s mind wins.
3. Is it easy for people to tell their colleagues or friends? Can they try it out immediately?
The moment right after “word of mouth”, when someone just hears about your product from another person, is the best time to capture interest. They should be able to try it out right away.
4. How long is the time to value or AHA moment? Can users use it in seconds, minutes, or days and weeks?
The shorter the time to value, the more likely for users to fall in love with the product.
5. What is the frequency of use? Is it daily, weekly, monthly, or annually?
The more frequently they use the product, the more likely a habit will be formed, leading to higher stickiness.
6. Who are the players right now? How big are they?
This is a tricky one. Without competitors, I’d question the demand for this space. With fast-growing competitors, I’d question the disadvantage. With a number of companies around 20-30 employees, I’d question the growth potential of the space.
7. Can I build a 10x product in this space? Do I and the co-founders have the background and expertise to create a significantly better product than everyone else?
If not 10x better, then there is no point in working on this idea because it is not convincing enough to acquire users. A 10% better product will have a super hard time asking users to form new habits.
8. How easy is it to get to the target audience? How big is the market?
E.g. restaurant is a terrible audience to go after. (See here) If it requires salespeople to go door-to-door to get to the audience, the idea won’t fly. Also, after focusing on a niche audience, it must be possible to expand to other groups.
Over to you
As a founder, it is your role to evaluate an idea and the relevant hypotheses before wasting resources on building it or even just researching it.
Assume -> Validate -> Build -> Feedback. Repeat.
And now with the number of #NoCode products out there, it is so easy for founders to run validations quickly.
Of course, after the hard lesson here, we pivoted to a product that was part of the global trend, easy to use, fast to spread, and quick to value. Toasty is now a video meeting platform focused on collaboration and engagement and is getting a lot more users than the matchmaking platform. As a side note, I stepped down from my CEO role in Oct 2020 and Eric Tsang is now leading the team.
To wrap this up, I’ve put these 8 questions onto this Google Sheet so you can clone and answer them honestly about your idea and see whether you still want to pursue it afterward. Click here to download it.
Here's What You Can Do
Grow your community of raving fans in 5 mins every week
Here's What You Can Do
Grow your community of raving fans in 5 mins every week