In my article 10 Commandments of Product Discovery, I wrote about the foundation of creating good products and services being a creative process called Product Discovery. This process aims to deliver a functional product to users in the shortest time possible, with the greatest certainty that the product team will achieve the originally intended goal and that the solution will be technically feasible.

I mentioned Discovery Backlog, which is a conceptual repository of ideas, opportunities, and issues ready for the product team to validate – meaning to assess which ones are relevant and should be included in the product roadmap. So, how does one go about building such a repository of ideas?

How to fill Discovery Backlog

Idea generation techniques are divided into quantitative and qualitative based on the type of data they rely on, and into generative and evaluative. Generative techniques help us find new ideas, while evaluative techniques help us assess the current situation.

Let's illustrate this with an example. I have an ecommerce store with the goal of increasing the shopping cart conversion by 1 %. I choose techniques from the four quadrants based on what I want to discover. The best results come from combining techniques from all four quadrants. Techniques from each quadrant can provide the following types of information:

Quantitative – Generative techniques:

  • I can ask all customers who place an order for feedback and thus obtain the CES (Customer Effort Score) of the purchasing process, that is, how difficult it was for them to complete the task. For example, I will ask: "How difficult was it for you to place the order?"
  • How to do it: Survey integrated on the thank you page after the order

Qualitative – Generative techniques:

  • As a customer leaves during any step of the purchasing process, I ask them through the Hotjar tool what they would improve.
  • How to do it: Hotjar survey triggered when a customer leaves the page

Quantitative – Evaluative techniques:

  • For the customers who completed an order and those who did not, I compare the conversion rates of individual steps in the purchasing process. I identify where most customers drop or where the greatest potential to increase shopping cart conversion is hidden.
  • How to do it: Data analysis based on data from Google Analytics

Qualitative – Evaluative techniques:

  • I conduct a usability test of the purchasing process.
  • How to do it: Usability testing of the website

Techniques for seeking ideas

Surveys

A survey is a quantitative technique primarily categorized as generative, but it can also be used to evaluate the current state. The advantage of a survey is that it allows you to gather a larger amount of data in a relatively short time.

Desk research

During desk research, you delve into your own knowledge base or public sources (internet, libraries, archives…). It provides you with both quantitative and qualitative data. Beware, don’t use desk research as a substitution for research with your own customers.

Get ideas from relevant and trusted sources only. Also, consider their timeliness as customer behavior can change over time. Interesting sources of information include archives in the field of social and behavioral sciences or open databases of scientific articles from social science journals published in countries of Eastern Europe.

Individual or group interviews

There is nothing more important than interviews with real customers. These can be conducted either individually (problem interviews) or in groups (focus groups). Before starting the actual qualitative interview, always make sure that you are talking to your real target customers through screening questions.

In interviews, try to capture stories. Don't hesitate to ask "Why?" to uncover respondents' real needs. In group interviews, use an experienced facilitator who can address the risks of groupthink and dominant participants. Our User Interview Cheatsheet will help you ask the right questions the right way.

Competitive analysis

Competitive analysis is a qualitative generative technique that helps you understand how other organizations solve a particular problem. It also helps you discover what can be done better, to gain a competitive advantage.

Data analysis

Analytical work with data is a quantitative evaluative technique. The conclusion of each data analysis should be the interpretation of the outputs, ideally with recommendations. The biggest challenge of data analysis is that either companies don't have such data or it's challenging for the product team to obtain it. However, it can provide very useful insights and even show that some ideas are not worth pursuing because there is no real customer problem or no opportunity arises from them.

If you don't have user and customer behavior data, map out two to three most important customer journeys, identify the key metrics, and implement your findings. The conclusion of each data analysis should include interpreting the outputs with any recommendations. Google Analytics can be a good starting point for websites and applications. To understand more advanced product analytics, I recommend reading the Lean Analytics book.

Usability testing

Usability tests are an evaluative qualitative method where you observe customers using the product. It evaluates the user's ability to complete the assigned task or the degree of difficulty and frustration experienced while doing so. In summative testing, you only need to observe six users to identify 80 % of serious usability flaws.

From collecting to exploring ideas

By combining qualitative and quantitative methods, the product team gathers a lot of ideas that can be validated during Product Discovery. Simply describe them briefly and add them to the Discovery Backlog. At Lighting Beetle*, we use the Idea Validation canvas. In the industry, you might also encounter terms like Business Problem Statement or Opportunity Canvas.