Our real-world testing was highlighted in the Harvard Business Review
In innovation and new product development, bad ideas are not the problem. The problem is the ability to discern good from bad.
Product development brainstorms can feel a little like trying to select a good apple from the display at the grocery store. You throw out a bunch of business ideas meant to generate double-digit growth. You crowd the table with ideas. Then, with a pile of ideas on the table, you try to identify the one that will work. If you’re an optimist, you’re looking for the good apple. If you’re a pessimist, you’re just desperately trying to avoid the bad apple.
Too many growth strategies—new product ideas, rebrands, revamped concepts—fail because they were developed and released based on hunches, groupthink, or incomplete market research like surveys, interviews, and customer feedback. Instead, innovation teams should consider real-world concept testing for idea validation before they launch—or even before they build. It’s a fast way to sort the good and bad apples in a way that is free of the usual bias.
Not all concept testing is alike. Some concept testing takes place with audiences who know they are part of a test; that knowledge tends to skew results. Heat-testing is concept testing that delivers real-world, behavioral validation and fail-proofs your growth strategy so it’s guaranteed to work.
First, let’s talk a little bit about what we mean by “concept testing” and “validation” when we talk about heat-testing. In heat-testing, concept testing is the process of taking a handful of ideas and using multivariate testing to validate which concept is worth pursuing and developing.
How does it work? Heat-testing is concept testing that uses real customer data and user behavior—clicks and signups on real ads in the wild—to validate demand for a concept.
Why multivariate testing and not just AB testing? Multivariate testing is a testing method that allows you to test multiple concepts, isolate variables, and provide validation beyond the capabilities of simple AB testing. It’s powerful because multiple tests take place simultaneously so that you learn quickly about the role of each variable in your concept’s success.
Because heat-testing uses real-world ads to represent concepts and strategies, it delivers concept validation with statistical validity. It’s the proof of demand all innovators are after: the confidence that real users actually want your product concept.
This is why bad ideas are not the problem. The problem is that too little due diligence is paid to the process of discernment and validation that comes between brainstorm and launch. It’s the act of concept testing, discerning which ideas or concepts are ‘good’ and which are ‘bad’—the picking up and inspecting the apples, in our metaphor—that makes all the difference between a launch success and a launch failure.
It all starts with an idea for a new product. Let’s say, for example, that you have an idea for dissolvable drink tablets for delicious beverages. Yes, the concept has been done before, addressing customer needs such as electrolyte replenishment and supplement delivery, but you’ve identified a new opportunity. The idea is just an idea at this point—absolutely zero actual product development, formulation, prototype, or minimum viable product exists yet. The goal of heat-testing? To use customer engagement to identify whether the concept is worth developing.
One of the biggest principles of heat-testing is that you never test a single concept—you test more than one at a time so that you learn as much as possible about which attributes of a concept drive interest. In the case of dissolvable drink tablets, we’d workshop to figure out a few different concepts to test. One might be focused on sustainability. Another, focused on non-alcoholic party drinks. And another? Focused on artisanal, elevated flavors. Boom, you’ve got three product concepts and you’re ready for a concept test.
Next, we’d develop our concept test framework. With three distinct product concepts in place, we’d start thinking about target audiences for our ad campaigns. Who are our potential customers? We’d conduct user research to brainstorm potential users and develop a few distinct target audiences for testing—one being an audience of sober-curious customers, another would be people who show interest in sustainable food and beverages, and a last might be the customers who are generally willing to spend a little more on delicious drinks (think: kombucha drinkers). For the validity of the concept test, we’d minimize the overlap between target audiences as much as possible.
Then we’d start designing ads and landing pages that express each product concept clearly and distinctly and provide a seamless user experience. Ads are shown to target customers as they scroll through Instagram. If they click, they end up on a landing page that says “[product] is coming soon—sign up to receive an email when we launch”. If they leave an email address, that’s validation of demand for the concept. Once each ad has been seen by a statistically valid sample of each audience, you have results you can trust.
Because it is both “real-world” and statistically valid, heat-testing is a form of concept validation testing rooted in real, objective evidence. In heat-testing, users encounter an ad and behave naturally, providing real validation rather than hypothetical insights. Product validation is drawn from real data. The data drawn from concept testing for validation is useful in a number of ways. It can be used to present a case for product development to stakeholders and product managers. Cost metrics, such as cost per click and cost per sign-up, can be used for modeling and planning.
So, it’s not bad ideas that are the problem. It’s the steps you take between ideation and launch that really count. Concept testing and validation through heat-testing is the safest, fastest way to validate demand for a product and de-risk your product development investment. Heat-testing identifies both good and bad ideas, putting you on the path to a winning growth strategy.
