There’s a lot of writing about the Lean Startup, Validating Assumptions, and Hypothesis Testing. Unfortunately, when implemented, it can become very confusing and is therefore often done incorrectly.
When I work with teams to become more lean, they often think that the best Lean Hypothesis is “We believe that our users need the thing we’ve designed. We’ll know we’re right if we build it and they use it.” This is just one of many ways Lean gets misinterpreted.
Today, let’s explore validating assumptions.
Before you decide what to build think about what assumptions are being made. After you’ve identified them… VALIDATE!!!If you’re a higher level manager, you should always ask,“What decisions associated with the product have been tested?”This will allow you to know how many assumptions have been validated in steps like the below.
A typical user story deconstructed https://marcabraham.wordpress.com/2016/02/18/my-product-management-toolkit-5-assumptions-and-hypotheses/
How it use the assumptions and experiments below:
Figure out which kind of assumption you have.
Conduct an experiment like the one listed to see if you were correct.
If your team’s assumption was correct, move forward to the next assumption.
If your team’s assumption was disproven, evaluate other options
TRY AGAIN
Assumption 1: We think this is a problem
Experiment- Research whether there is a problem people are talking about and if there’s an existing solution. (Google, Twitter, and Quora can be very helpful)
Assumption 2: We think this is a problem for X group and there are a lot of them and they all have the same problem.
Experiment- Do some demographic research about how many people are really in that group. Then ask some of the people in that group what their experience has been and see if they mention the problem. If they all agree, you’re good to go. (Census data and User interviews)
Assumption 3: We think this is a solution for the problem
Experiment- Sketch the solution and talk to some potential users, make sure they think that your solution will help. (Get out of the building!)
Assumption 4: We think X group will pay for this solution
Experiment- Ask potential customers how much they’d pay for the solution. (Price before Product, Period.)
Assumption 5: We think this solution is feasible
Experiment- Chat with your engineering leads about what you’re thinking about building and that it’s not impossible or super hard to do. (They’ll appreciate being involved early on.)
Assumption 6: We think adding Z feature will add a lot of value
Experiment- Interview customers/users to find out whether the product is helpful enough without the feature or if it’s critical to include. Alternatively, create a landing page with and without that feature listed and see what conversion rate is. (A/B test without and without the feature in your mocks)
Assumption 7: We think what we designed is usable to solve the problem
Experiment- Create a paper or clickable mock and ask users to complete the task, or better yet just see what happens without any prompt. (Invision, UserTesting.com, AlphaHQ, Validately can help you out)
Assumption 8: We think we can build this in Y time
Experiment- Get your development team into a room, break up the product into high level features, have them provide high level point estimates or T-shirt sizes to guide how complex what you wish to build will be and how long it will take. (It’s better if you can compare this to previous work but that isn’t always an option)
Assumption 9: We think if eliminate this feature X group will be OK
Experiment- Don’t ask the users if they’ll miss the feature! Show them the product without it and see if they complain. (Invision, UserTesting.com, AlphaHQ, Validately can help you out)
Assumption 10: We think what we’ve built so far is on the right track
Experiment- Bring in some REAL users to test it out; talk to some REAL customers about whether it’s valuable enough to pay for. (Don’t forget to ask OPEN ENDED questions)
Assumption 11: We think we’re ready to launch
Experiment- Test the product internally. Check in with Marketing and Sales to ensure they have what they need and that they understand the financial viability of the product. (QA is not always enough)
Assumption 12: We think what we launched is being used
Experiment- Look at your Google Analytics, Mixpanel, Heap.io or other event tracking tools. (These should be setup BEFORE launch)
Assumption 13: We think what we launched is being used to solve the problem
Experiment- Go back to your target users and see how they are using the tool you’ve built. Talk to random other users about what they are using it for (You may learn that there is an additional market)
Assumption 14: We think we’re missing Z feature
Experiment- Return to Assumption #6.