Assumption / Validation Flowchart

There’s a lot of writing about the Lean Startup, Validating Assumptions, and Hypothesis Testing. Unfortunately, when implemented, it can become very confusing and is therefore often done incorrectly.

When I work with teams to become more lean, they often think that the best Lean Hypothesis is “We believe that our users need the thing we’ve designed. We’ll know we’re right if we build it and they use it.” This is just one of many ways Lean gets misinterpreted.

Today, let’s explore validating assumptions.

Before you decide what to build think about what assumptions are being made. After you’ve identified them… VALIDATE!!!If you’re a higher level manager, you should always ask,“What decisions associated with the product have been tested?”This will allow you to know how many assumptions have been validated in steps like the below.

How it use the assumptions and experiments below:

  1. Figure out which kind of assumption you have.

  2. Conduct an experiment like the one listed to see if you were correct.

  3. If your team’s assumption was correct, move forward to the next assumption.

  4. If your team’s assumption was disproven, evaluate other options

  5. TRY AGAIN

Assumption 1: We think this is a problem

Experiment- Research whether there is a problem people are talking about and if there’s an existing solution. (Google, Twitter, and Quora can be very helpful)

Assumption 2: We think this is a problem for X group and there are a lot of them and they all have the same problem.

Experiment- Do some demographic research about how many people are really in that group. Then ask some of the people in that group what their experience has been and see if they mention the problem. If they all agree, you’re good to go. (Census data and User interviews)

Assumption 3: We think this is a solution for the problem

Experiment- Sketch the solution and talk to some potential users, make sure they think that your solution will help. (Get out of the building!)

Assumption 4: We think X group will pay for this solution

Experiment- Ask potential customers how much they’d pay for the solution. (Price before Product, Period.)

Assumption 5: We think this solution is feasible

Experiment- Chat with your engineering leads about what you’re thinking about building and that it’s not impossible or super hard to do. (They’ll appreciate being involved early on.)

Assumption 6: We think adding Z feature will add a lot of value

Experiment- Interview customers/users to find out whether the product is helpful enough without the feature or if it’s critical to include. Alternatively, create a landing page with and without that feature listed and see what conversion rate is. (A/B test without and without the feature in your mocks)

Assumption 7: We think what we designed is usable to solve the problem

Experiment- Create a paper or clickable mock and ask users to complete the task, or better yet just see what happens without any prompt. (Invision, UserTesting.com, AlphaHQ, Validately can help you out)

Assumption 8: We think we can build this in Y time

Experiment- Get your development team into a room, break up the product into high level features, have them provide high level point estimates or T-shirt sizes to guide how complex what you wish to build will be and how long it will take. (It’s better if you can compare this to previous work but that isn’t always an option)

Assumption 9: We think if eliminate this feature X group will be OK

Experiment- Don’t ask the users if they’ll miss the feature! Show them the product without it and see if they complain. (Invision, UserTesting.com, AlphaHQ, Validately can help you out)

Assumption 10: We think what we’ve built so far is on the right track

Experiment- Bring in some REAL users to test it out; talk to some REAL customers about whether it’s valuable enough to pay for. (Don’t forget to ask OPEN ENDED questions)

Assumption 11: We think we’re ready to launch

Experiment- Test the product internally. Check in with Marketing and Sales to ensure they have what they need and that they understand the financial viability of the product. (QA is not always enough)

Assumption 12: We think what we launched is being used

Experiment- Look at your Google Analytics, Mixpanel, Heap.io or other event tracking tools. (These should be setup BEFORE launch)

Assumption 13: We think what we launched is being used to solve the problem

Experiment- Go back to your target users and see how they are using the tool you’ve built. Talk to random other users about what they are using it for (You may learn that there is an additional market)

Assumption 14: We think we’re missing Z feature

Experiment- Return to Assumption #6.

Classic MVP Types

Thanks to the popularity of The Lean Startup, the term MVP (Minimum Viable Product) is now commonplace in the startup ecosystem. Unfortunately, it is often misused and the products built are either not Minimum or not Viable or worse not Products.

An MVP is supposed to be the smallest thing you can deliver that provides value to the user and allows the creator to learn something. One common mistake when building MVPs is insisting that it “needs more features”. Another is not setting it up in a way that metrics can be measured for learning.

So, how do you setup a good Minimum Viable Product so that you can learn about your users and customers. Here are a few MVP types to get you familiar with the basic options and well known companies that utilize the technique.

Wizard of Oz: “Pay no attention to the man behind the curtain”

The user experiences what seems like a magical technical interface that does things for them, when in fact there are humans hidden in the background doing the work. Zappos started with practically no inventory and purchased the shoes after customers bought them online before shipping. This allowed them to learn about how shoe shoppers bought online and what was important to them.

Concierge: “Let me take care of that for you”

Often confused with the Wizard of Oz, in this MVP the human company representatives are known to the users. They speak to them on the phone or email back and forth with customers. This option allows a startup to learn by listening directly to the people using their service. StitchFix started with the founders getting on the phone and talking to people ordering clothing through them.

Video: “Watch this video of how we’re going to change your life”

Sometimes you can explain what your product will do without having it work just yet. DropBox is most famous for this (watch the original video), but most crowdfunding sites utilize videos too. By showing the users what the experience will be, and gauging their interest, you learn what is most important to build.

Email digest: “I’ll bring the news to you”

Rather than creating a big content site, start by generating an email that is sent on a regular basis. This gives you a defined time to iterate on the format & content in the email and provides small feedback loops every time the email is sent out. ProductHunt started as a daily email with a single product, then it became lists in the email, and then an entire site defined by what users found engaging.

Landing Page: “Here’s a list of things we’re planning on doing for you”

By far the simplest MVP to create thanks to tools like SquareSpace, Instapage, QuickMVP and others. It’s a small website that uses text and potentially a few images to talk about what the company will deliver. Similar to the video, you’re gauging interest and attempting to validate your value proposition with potential customers, but it’s a lot easier to update a line of text than captured video. General Assembly utilizes landing pages for new courses they are considering launching. The page has a “Join Waitlist” button instead of one to register and if the waitlist is large enough, they produce the course.

There are countless other examples of successful companies that have launched an MVP to learn about their market before building a full blown product. What will you do to validate your assumption that your product is in high demand, and how will you measure it?

Want more women in tech? STOP hitting on us!

Disclaimer: This post is not a personal story, or even a story about one person I’m friendly with. I have never been hit on by a VC, founder, boss, or coworker, but I’m sick of hearing about others who have been degraded by males in power.

Why am I writing this post:

Over the past year and a half I’ve watched a close friend try to break into the the startup tech industry with moderate success. Everyone that meets her knows that she’s awesome and she’s been hired on contract a few times.

Recently, she decided that the universe was telling her to go back to the corporate world. What was the straw that broke her back? Was it the lucrative paychecks or spending accounts? No.

All it took was a founder she was interviewing with asking her out on a date.

When I heard this, I was furious. How could a founder do that? Don’t they realize that they are alienating the candidate? Didn’t they think there would be a backlash? Have we learned nothing about how to treat women in the workplace in the last 50 years?

It was then that I realized that she had no recourse. If she called out the founder publicly for his inappropriate actions, she’d get branded as a trouble maker and technically there was nothing illegal to report.

I decided to write this post as a response. I’m well known in the NYC tech ecosystem for promoting diversity and supporting women. I’ve been called into some very large companies to discuss what can be done to improve diversity. This is part of my message to those and other companies. Start by stopping men from hitting on your female employees.

What I learned (that everyone should know):

My first step was requesting more stories. I’m not an investigative journalist and this post shouldn’t be analyzed as such, but what I learned in a few short days was startling. As a data nerd I am aware that there is no way that this is representative of the population to a scientific standard. But even from that small sample size, I received dozens of stories. And many people in my network reported stories from their own circles.

I reached out to a women in tech email list that I’m a member of and posted on my Facebook asking a simple question “I’m writing a post about women getting hit on in tech by founders and how it isn’t helping diversity numbers. If you have anything to share, send me a message. All stories will be kept private.”

I learned of the countless times women had been hit on by their bosses, co-workers, or men they were attempting to do business with. I heard about investors that turned pitches into opportunities to get laid. I listened to recounts of women where lunch with male coworkers included crude stories of sexual escapades. I cringed at stories of women inventing boyfriends to ward off unwelcome advances as coffee meetings were relocated to bars.

Most disgusting of all was an anecdote that was forwarded to me about a startup whose parties at the boss’s house had a NO CLOTHING RULE after 10PM. I was so shocked. I thought I had misread the message.

What the problem is:

You maybe saying to yourself, there’s nothing illegal about the snippets I’ve shared above. You’re right, consenting adults can take part in activities that they choose to. And asking someone out once isn’t often considered harassment because it isn’t severe or pervasive.

But this post isn’t about what’s legal, it’s about how hitting on women makes them leave your organization, and often the tech industry in general. Of the women I asked, many of them left the company within months of the incident. Some stopped the interview, pitch, or professional relationship immediately.

What about asking out an attractive person of the opposite sex who shares similar passions for technology on a date is problematic? If you’re on a dating app, probably nothing. The same goes for a singles event.

Additionally, we’ve all heard stories of loving couples that met at work. Someone had to make the first move right? I don’t know the right answer to that question, but maybe waiting for the woman to make it obvious that she’s interested is the key. A misstep in that realm can make many future interactions tense.

However, if you have a potential professional relationship with someone and there’s an imbalance of power, you’re skating on thin ice. Many of the stories shared with me included later feeling that people weren’t treated as well or given as many opportunities if they rebuffed the advances. This is where founders have the biggest challenge.

Even if YOU would never retaliate because of a rejection of your proposal, the recipient doesn’t know that. What you think is harmless flirtation, may make others assume something more is going on and worse think the woman is advancing her career in unprofessional ways.

Remember, to them it may not be a compliment; you are potentially making them feel uncomfortable. Would you want to work in a work environment that wasn’t comfortable?

Women want to be invited to join companies or receive investment in their ventures because of their merit, not because they put out. By objectifying women you are degrading us. I assume the same goes for men. There goes that equality thing rearing its head.

Also, I know this issue isn’t one-sided. A male friend of mine reported to me that he was once hit on by a female potential customer. She offered to expediting the sale if he was agreeable to her demands. These anecdotes appear to be more rare.

Though I’m not an investigative journalist, I’ve never known a man to change the way he dressed at work because of the advances and comments coworkers frequently made. A number of women shared that they don’t wear makeup, heels, or skirts to work because they want to avoid attention being drawn to them. They are uncomfortable in the environment they spend at least 8 hours a day.

What to do about it:

If you think these actions aren’t causing problems for your company, think again. The negative impacts of a less diverse workforce are seen in employee retention and overall financial performance. Asking female staff out seems like a small thing to avoid in order to reap the benefits of them staying engaged at work.

In conclusion, I’m sure that many people who read this post would never make a sexual advance on a coworker or candidate. That’s awesome! Please share with someone who might not already be on the same page. Also, check out these other microaggressions you maybe guilty of that make women leave tech companies.

Comments are welcome below, but if you have a private story to share, please email me at tami (at) tamireiss (dot) com, I promise to listen.

The XP Agile Approach in Plain English

BETTER APPROACH = BETTER SOFTWARE = BETTER OUTCOMES

XP and other Agile methodologies bring about a better software product and overall better business.

XP or Extreme Programming is about solving problems quickly and writing maintainable code. Below are some XP Agile practices that I’ve found work well with XP teams explained in PLAIN ENGLISH.

KAIZEN (改善)

The Japanese principle of continuous improvement — a concept that not only makes things better, it makes people better. Employ it in everything you do from product feature MVP development to internal process improvements.

INCREMENTAL REQUIREMENTS

By working in small batches, programmers can more quickly implement their work, demonstrate how it works, and make it available to the wider team faster and more efficiently.

FULL STACK STORIES

Don’t build buttons that go to no where or backend processes that aren’t used, build both parts at the same time to ensure the full user experience works.

TEST-DRIVEN DEVELOPMENT

Create a safety net around the code base by writing automated tests that verify feature completion and that nothing else has been broken. Write the tests before writing any production code. If the tests aren’t “green” when they are run, the feature isn’t complete.

CONTINUOUS INTEGRATION

No one works in a silo. Multiple times a day, developers integrate their work with the larger team, to make sure that everyone is always working on the latest update. This means that there is no more “end of project” surprise when all of the code gets put together.

REFACTORING

Developers are encouraged to simplify existing code whenever possible — for a better and easier-to-modify code base. (It’s like cleaning out the rotten veggies out of the refrigerator drawer before putting fresh produce in.)

COLLECTIVE CODE OWNERSHIP

Leverage the brain trust of your entire team by allowing everybody to extend or improve every developer’s code. Bottlenecks are eliminated and better code is crafted because everybody is involved.

PAIR PROGRAMMING

Two programmers working together at a single computer. It’s better for knowledge-sharing, increased productivity, and higher quality output. It also helps with onboarding and lack of going down rabbit holes (see graphic on top for what this looks like)

INFORMATIVE WORKSPACE

To avoid project progress bogging down, fill developer workspaces with “information radiators” — useful charts and diagrams that keep everyone up to date on the health of the project at a moment’s notice.

JUST ENOUGH DOCUMENTATION

Often documentation standards are out-of-date minutes after they’re, well, documented. And they slow everything down. Minimize documentation before implementation, and focus on it at project end for greater efficiency and accuracy.

SHORT STANDUP MEETINGS

No one likes meetings because they have a tendency to go on and on. Limit standups to five or ten minutes, and require everyone attending to stand. This ensures highly focused discussion of progress, health and upcoming coordination — so everyone can get back to work.

ITERATION RETROSPECTIVES

Every step of the project, the team comes together to discuss the tactics that worked well — as well as those that need improvement — to insure that everybody is continually improving.

PROJECT RETROSPECTIVES

Hold a longer reviews of all the issues and ideas we’ve uncovered to make the most of every project moving forward.

What I learned from Textio about Job Postings

I’ve used Textio.com a few times for roles and today I finally got a score of 100!

Here’s what I learned…

My original job posting:

About VenueBook:

Founded 6 years ago, VenueBook is poised to become the default way the corporate event planners book venues online. We have over 500 spaces managing their events on our platform and a marketplace for event planners that serves thousands of users every week. Our team is 30 people strong with most of us based at our NYC headquarters on Houston and Thompson and remote team members growing our presence in Chicago, DC, and SF markets. VenueBook is well-funded, has product-market fit, and will be raising a Series B early next year to accelerate our growth.

About the Product Manager Search / Discovery Role:

Helping event planners find the perfect space for their parties and presentations is the top of the Venuebook funnel and this role will be dedicated to making that an awesome experience for those users.

Guided by user research and customer development, you’ll work with our UX designer to define the user flows that will make it easy and fast for someone to review and select a venue. You’ll get to play with Optimizely, Google Analytics, and Mixpanel to use data to drive your backlog. Our tech team will collaborate with you to get changes out the door ASAP and drive results.

This is a growth position. Over time, you’ll get exposed to other parts of the VenueBook platform and work with our Product Lead to choose which other areas you’d like to own.

About You:

  • You’ve got 2+ years Product / Project management experience on an Agile team

  • You have a passion for data driven decisions and improving KPIs

  • You are looking to grow your skills with modern PM + UX tools

  • You love talking to users and making their jobs easier through technology

  • You’re excited to work closely with marketing, design, and engineering

  • You’ve been looking for an opportunity to have impact at a growing company

About the Interview:

You’ll have the opportunity to speak with members of the product, tech, marketing, and executive teams. Some interviews will be over hangout, others will be in person. We look forward to meeting you!

Textio Score > 74 (above average) with Neutral tone


Step 1: Fix the big things that were missing

I included an equal opportunity employer statement (+22 points!)

Step 2: Improve the language

Replacing drive results with GSD and KPIs with important metrics(+2 points!)

Step 3: Add bulleted content

Texti.com has found an ideal balance for percentage of a job posting that should be bulleted vs paragraph.

But when I added the bullet, I used a male candidate leaning phrase! (+0 points)

After changing that to “in stressful situations” I was 1 point away from a perfect score… I knew I could do it.

But they had run out of advice, what should I do?

Step 4: Make equality a priority

I made equal opportunity employer statement stand out with a ** (later we made it italics) and wallah 100 points!

Our final job posting:

About VenueBook:

Founded 6 years ago, VenueBook is poised to become the default way the corporate event planners book venues online. We have over 600 spaces handling their events on our platform and a marketplace for event planners that serves thousands of users every week. Our team is 30 people strong with most of us based at our NYC headquarters in SoHo and remote team members growing our presence in Chicago, DC, and San Francisco. VenueBook is well-funded, has product-market fit, and will be raising a Series B early next year to accelerate our growth.

About the Product Manager Search / Discovery Role:

Helping event planners find the perfect space for their parties and presentations is at the top of the Venuebook funnel. This role will be dedicated to making that an awesome experience for those users.

Guided by user research and customer development, you’ll work with our UX designer to define the user flows that will make it easy and fast for someone to review and select a venue. You’ll get to play with Optimizely, Google Analytics, and Mixpanel to use data to drive your backlog. Our tech team will collaborate with you to get changes out the door ASAP and GSD.

This is a growth position. Over time, you’ll get exposed to other parts of the VenueBook platform and work with our Product Lead to choose which other areas you’d like to own.

About You:

  • You’ve got 2+ years Product / Project management experience on an Agile team

  • You have previous eCommerce or marketplace product experience

  • You have a passion for data-driven decisions and improving metrics

  • You have a GSD attitude but remain calm in stressful situations

  • You are looking to grow your skills with modern PM + UX tools

  • You love talking to users and making their jobs easier through technology

  • You’re excited to work closely with marketing, design, and engineering

  • You’ve been looking for an opportunity to have impact at a growing company

About the Interview:

You’ll have the opportunity to speak with members of the product, tech, marketing, and executive teams. Some interviews will be over Google Hangout, others will be in-person. We look forward to meeting you!

VenueBook is an equal opportunity employer and does not discriminate based on gender, race, or sexual orientation.

Want to apply? Check out https://venuebook.com/about/jobs/