How to avoid product mistakes using hypothesis validation?

Why should you learn about validating hypothesis?

Developing a digital product is not an easy process and as a product owner or stakeholder you need to accept a certain risk that comes with it. Building and validating product hypotheses can significantly reduce that risk, so it prevents you from wasting valuable resources, mostly time and money.

In many cases, validating a product hypothesis requires gathering feedback from your target group. You can use various hypothesis validation techniques, e.g. user testing or interviewing potential users. Having information that comes directly from future customers helps the product team to make more accurate product decisions.

Validating product hypotheses - when done right - can greatly influence the future of your digital product and business. But it doesn’t come without effort. It requires some knowledge, investment, and time. Therefore, it’s important to learn how to do it correctly. Let’s equip you with the basics, and some examples of validating product hypotheses.

What is a hypothesis in digital product development?

A hypothesis is a specific kind of statement that includes a belief you may have about your users, market or product. When working on a digital product concept there are always some assumptions popping up in stakeholders’ heads. That’s a natural process and sometimes even happens unconsciously. These theories can be used to formulate product hypotheses.

A product hypothesis may consider:

• users’ level of knowledge,
• users’ tech skills,
• users’ expectations,
• users’ needs,
• product security,
• market demand,
• product usability,
• product features,
• text instructions, etc.

How to formulate a good product hypothesis?

Let’s look at the characteristics of a good hypothesis. The most important is to keep the statement clear and simple - it must be understandable to you and your development team. Use terminology known by all the parties involved in the process.

Secondly, the hypothesis needs to be relevant, direct and specific to the product problem (it shouldn’t include any generalizations). Such a hypothesis can only be stated after detailed observation of the product (if an MVP version exists), users’ behavior, market trends, competition, etc.

Formulating a product hypothesis starts with doubting your assumptions. You need to know which of your thoughts are facts and which ones are just beliefs. Start with separating the two and writing down your beliefs - you can then use them to formulate product hypotheses.

Here are some examples of product hypotheses, which were stated for products recently developed at Boldare:

Example 1: Small e-commerce businesses from Poland will use our design tool.

Example 2: Users are able to go through all the steps of the Progress Guide (initial process for new app users).

Example 3: Users are able to fill in the data in the education verification process form.

Later in this article, we describe how we validated the above hypotheses. Spoiler alert: validation was negative, but it turned out to be a very positive result. How come? Read on to find out.

Hypothesis validation process

At Boldare, we use test cards to have a clear vision of what the hypothesis is, our plan to verify and measure it, and what outcome will confirm it. The template consists of four unfinished statements that need to be completed. Here they are:

1. We believe that…
2. To verify that, we will…
3. And measure…
4. We are right if…

At this point, it’s crucial to decide on methods that will help in verifying the hypothesis, and choose testing techniques for checking if the statement is true. It’s necessary to set a specific outcome that will indicate whether the hypothesis validation was positive or not. Let’s see some examples.

Case story 1: Verifying market needs

Last year, we partnered with a leading Internet printing house in Poland. Our cooperation was broad and consisted of a number of separate services and processes. Here, we’ll tell you one short story that involved testing a market demand hypothesis.

Our partner bought a license for a simple tool that enables the design of promotional materials, like flyers, business cards or posters. They wanted to implement the tool in their web application as they believed this is something their users would value.

We had one market segment to be checked: small e-commerce businesses (other segments were by default not a target group for this tool). We set our hypothesis as follows: Small e-commerce businesses from Poland will use the design tool. To verify that we created a list of questions and interviewed a number of e-shop owners.

It turned out that they don’t really need a tool, as they already hire designers who know their brand and company’s visual identity, and who prefer to use more advanced design tools. Because the hypothesis had been validated negatively, we recommended our partner not implement the tool and not invest in it any further. It just wouldn’t pay off.

Case story 2: App onboarding

This is a story about users onboarding to our partner’s mobile application. The app was created to serve leaders who want to bring more awareness into their professional and personal life. The first steps in the app were supposed to be easy, helping leaders register and understand how they can use the various product features.

For this, the development team created a special app section called the Progress Guide. After entering this section, users were guided through the eight necessary steps of onboarding with instructions and explanations. The team’s hypothesis was that the Progress Guide is well written and designed, so all users will go through it smoothly.

To verify the hypothesis, the product team tested it LIVE with real users on a full running version of the app. It turned out that most of the users were already stuck at the first step of the Progress Guide and didn’t know what to do next. This analysis showed the team how they could develop the section and which elements required improvement.

Case story 3: Are the instructions clear for users?

Some time ago, we developed a product for one of the Saudi government ministries. During the process, we were building multiple hypotheses and validating them. The app’s purpose was to gather and verify information about users’ education. Here is one of our hypotheses: Users are able to fill in the data in the education verification process form.

To verify this hypothesis we asked users to fill in the form according to the included instructions and send it to us. By doing so, we measured how many users struggled with this task and which fields were problematic for them. We set our hypothesis verification as follows:

We are right if 90%-100% users fill in the app form accurately. Our testing group consisted of potential users - we chose participants with the characteristics of the product’s target persona.

This case story confirms that negative validation of a hypothesis can positively influence a product. We discovered that some users did not understand all of the instructions. They did not know what information they should enter in a particular field. Knowing that, we could improve the instructions and save precious time for our partner (who otherwise would have had to ask users for the missing data, taking more time). After applying improvements, we set another hypothesis and repeated the process until we got our hypothesis validated positively.

When to use hypothesis validation?

You can set and validate product hypotheses at each stage of your product development. At Boldare, we work within a process of full cycle product development that consists of four phases: prototype, MVP, product-market fit, and scaling. We validate various product hypotheses in each of these phases. We never base our work on assumptions or intuition, we prefer to check everything at each step of product creation and base all our product decisions on facts. It helps to avoid costly mistakes and prevent product failures. But above all, it enables us to build high-quality products that users understand and want to use.