In this excerpt we change the way you think about the formulation of hypotheses to expedite your product creation process. Where we used to think of hypotheses as scientific fodder, our book transforms the definition and formulation of hypotheses into something we can use in our day-to-day product efforts; proving and disproving our assumptions so we can make quick product decisions and continue our forward momentum. Check it out.
The following excerpt is from Groundwork: Get Better at Making Better Products by Vidya Dinamani and Heather Samarin.
We are constantly inspired with new product ideas—whether that’s through customer observations, the competition, or an “aha!” moment in the shower. Every idea has the potential to be formed into a hypothesis. The way to ensure you have a strong hypothesis, one worthy of testing, is by reviewing that they conform to these six characteristics:
This seems obvious, but think about the number of times someone has a good idea on the team. Good ideas are a dime a dozen, sadly. We’ve seen multiple teams become overwhelmed with new ideas entering the system because they see a competitor launch a new feature, or because an investor or senior leader gets inspired by the latest shiny object. Being logical means that there is clear, sound reasoning connected to the hypothesis. The idea doesn’t come from left field; you can point to evidence to suggest the outcome will be promising.
Making sure a hypothesis is testable is one of the hardest aspects of generating a great hypothesis. Often, teams feel like they need to fully build and launch their product to get actionable data so they forgo hypothesis testing altogether. While your results may not be statistically significant, think about how you might, with minimal resources, prove that you’re heading in the right direction. With practice, writing a testable hypothesis becomes easier—teams get very creative and quickly adapt to creating small tests with minimal resources.
How you put together a hypothesis counts. Using deliberate words that indicate a specific outcome is important. Otherwise, any given outcome can be seen as either confirming or falsifying. Being precise means being crystal clear on what you’re testing and why, and documenting the rationale for what it means to run the experiment, what resources you’ll need to experiment, how long it will take, and when you’ll know the results.
About Something Measurable
You need to understand which numbers matter before you do the research. If your hypothesis is that more visitors will see your webpage when you make a certain change, then you need to know exactly how many more must visit for the change to be meaningful, and thus, for the hypothesis to be confirmed. An increase of 1% may mean nothing to one company, and it may mean millions in incremental revenue to another. Understanding what measures are important gives you a good indication of whether the experiment or test should even occur. If the results can’t confirm or falsify your hypothesis, why do the research? This is a hard characteristic to pin down, but it’s critical to ask both if you know what the measurement is, and whether the measurement is meaningful.
Has an Expected Outcome
We want you to be able to explain with a logical, clear reason what outcome you expect. The line gets tricky here—if you already know the outcome, then go get the data to prove how you already know this. Why spend time testing things you already know? We see teams share hypotheses with us all the time about things they already know. They do this because it’s comforting to be proven right. On the other extreme—if you don’t know what outcome to expect, then how will you plan the precise experiment that could achieve that outcome and how could you know what to measure? Make sure your hypothesis specifies an outcome and that the outcome will address your hypothesis.
The last of the characteristics is making sure that you can invalidate your hypothesis. The way to think about this is to make sure you don’t set your research up in such a way that the results are subjective, or that you’re not capturing the right data to be able to have a clear result.
These six characteristics are intended to help you coach your product managers. Instead of just providing feedback on a hypothesis (or rejecting it outright), point to one or more of the guidelines and discuss how their hypothesis may miss the mark. It’s a teaching moment and a much more productive conversation at the same time.
Product Rebels is a product management training and coaching firm run by long term product executives for companies like Intuit and Mitchell International. We have trained over 200 companies, small and enterprise level, in the skills and frameworks that help product management leaders and product managers deliver kick-ass customer experiences. We have a passion for finding efficient ways of infusing customer insight into everything product teams do in pursuit of experiences that customers love …and that drive growth. Join us in the Product Rebels Community on Facebook or the Product Rebels Community on LinkedIn.
Take a look at our very practical training courses and coaching programs that give you practical tools, frameworks, and support you can use tomorrow in becoming a more effective product leader. www.productrebels.com