Determine research needs

While not exclusive to the Discovery and Definition phase of product life cycle, research cultivates unbiased understanding. For example, observing a field service technician using paper instructions revealed undocumented context.

Qualitative research intended to build effective products falls into the following four categories (arranged from most open-ended to most specific).

• Explorative: gaining a general understanding of the domain, mapping the business using service design.
• Generative: The process of researching and identifying new hypotheses for the business as well as building an understanding of the domain (e.g., “We can save costs if we automate 90 percent of loan underwriting”).
• Formative: The process of testing new hypotheses before and during delivery of a product. For example, testing a checkout process and quantifying a certain conversion rate.
• Summative: Performed at the end of the project to validate success; integrates into the established metrics for success.

Research enables the cross-functional team to build effective software and address potential objections. As a reminder, all participants are biased, and they each have an agenda, including the end user. Research is the best tool to eliminate bias and gain insight into what’s necessary.

Research falls into two categories:

• Qualitative: Gaining an understanding of a business through mapped workflows and identified system roles
• Quantitative: A/B testing, adoption through time, average customer value, etc.

A word of caution: there’s often resistance from clients when research is suggested as an activity. The design industry, and more specifically design agencies, have really mucked up the perception of what research is and what it produces. More often than not, the research such groups pitch and sell ends up being explorative, generating myriad ideas that don’t necessarily distill the data into an actionable set: activity without progress. So clients have the right to be cautious.

In the context of product and shipping software, research can be any or all of the following:

• Interviewing a user
• Documenting a workflow
• Testing a feature set with multiple stakeholders
• Observing people in their field of work
• Comparing multiple applications for their feature richness
• Testing two layouts of a form for higher conversion
• Creating a service map

A workshop with a client is a type of research that offers an understanding of the client’s domain. Ongoing research is conducted daily throughout the life of a product. Research coexists with software delivery and should be leveraged before, during, and after the product is shipped.

Exploratory research

Exploratory research is executed when the team needs to understand a problem more clearly before determining the approach. This informs the best investment of time in delivery by determining what is understood and what remains to be learned.

• Why to do it: During exploratory research, build an appreciation for the challenges the industry faces. It’s essential to learn from the products that have been built in the space to date. The successes and failures identified early through exploratory research build empathy with the client. Empathy is at the core of a thoughtfully designed product for both organizations and customers.
• When to do it: For every new domain, for every new client, and for every sales pursuit, invest in exploratory research. Without research to build on from previous initiatives, developing an organizational understanding in a new domain begins from scratch.

Exploratory research may kick off the following activities:

• Industry benchmarking. Present a comparison of various products and companies. Before starting, decide what qualities to measure against and identify an extensive list of companies to investigate.
• Competitive analysis. Analyze activities occurring across an industry objectively. Target vectors that threaten a specific product or business. Watch for outliers that do not share 100 percent of the product goal but have the potential to steal your lunch.
• Portfolio overview. Evaluate a product portfolio across a 2×2 diagram to define maturity and usefulness. Dive into additional heuristics across the applications and present an app-by-app findings comparison. Think of this as an internal competitive analysis.
• Stakeholder interviews. These are internally facing versions of user interviews. Often in a group setting, various politics and biases get in the way. By having structured one-on-one conversations, cut through internal noise, and discover a true consensus among your stakeholder group. This is useful when there is a misalignment in expectations or when the truth is different than what is being presented to the team.

Generative research

Generative research, generally considered a qualitative activity, is meant to generate new ideas, new understandings, and new directions. The goal here is to explore many possible vectors that focus on one core problem. Because of the divergent thinking here, it is important to have clear success criteria in place before embarking on this type of research.

As a product design team, we set ourselves apart from the industry by our focus on outcomes, not activity. Activity without progress is worthless. For every activity you pick, ask yourself how this is moving the team one step closer to building working software.

Why to do it: To become a strong partner with the business by engaging in their core business problems. This is often the healthiest form of up-front research possible for relationship-building and defining a product’s success. It is risky and gets caught up by the enemy of progress—innovation churn.

When to do it: These activities generally happen immediately following a workshop or major release.

Generative research may lead to the following activities:

• Design sprint: They are only productive for very healthy organizations with flexibility in budget, scope, and timeline. Design sprints are equally focused on building the thing right as they are on building the right thing. The business has a clear line to outcomes and the ability to be successful, often involving process changes outside software. Venturing into a design sprint may jeopardize momentum and lead to a lack of clarity if not handled by a seasoned facilitator. When considering this course of action, always engage with the MD and PDM before suggesting it to the client.
• Facilitated workshops: These are targeted collaborations involving the entire team. Do not go more than a month into delivery without finding an excuse to get the team together to collaborate on product direction and function. Even a half-hour session to ensure alignment is valuable.
• Contextual inquiry: Running a contextual inquiry requires a seasoned practitioner and a solid amount of preparation. For these efforts, go to the environment where the product is going to be used. The output for this type of research is typically a well-defined service blueprint.
• Card sorting: This is a well-established research technique for discovering how people understand and categorize information. The results are used to group and label information in a way that makes the most sense to your audience.

Before embarking on any research activity, be sure to clearly document the intended outcome as well as the impact of not doing it. If you cannot identify a negative impact of not doing the activity, then it is probably unnecessary. If you are trying to decide what to do next, ask another Devbridge team member for help. Managers, MDs, and practice leads are great resources.

Formative research

Formative research brings the qualitative and quantitative together by evaluating the business’s current state, identifying key areas of improvement (e.g., build a new product or feature enhancement), and charting the path forward. Formative research includes the following:

• Heuristic evaluation: Identify usability problems in the user interface (UI) design. Is there objective truth in design? When it comes to best practices and general usability standards, the answer is yes. Heuristics are the graded evaluation of the UI quality within a given application. This type of evaluation is most powerful when encountering an application to be upgraded rather than replaced.
• User interviews: It’s critical to get input from users, hearing directly from those interacting with the product or feature.

“Don’t take my word for it!” —LeVar Burton

Wise words. We hope to hear the same words from every stakeholder in our engagements. Surgeons cannot perform their own kidney transplants, and stakeholders should not try to be their own source of user context.

There’s no script to follow with research. Each activity has a unique output. That being said, the two types of research that are absolutely necessary for healthy product delivery are formative and summative research—how the thing works and whether the thing works as intended.

Summative research

Summative research demonstrates progress by learning about failures early and adapting to changing conditions with both the client and the users of the product. If we’re not doing this, we’re not making great things. Summative research includes the following:

• Usability testing. Paper prototyping, clickable prototyping, testing in development environments . . . Whatever it is, don’t wait until production to validate how people use the platform. Conduct this testing on the devices leveraging the product and simulate real-world working environments if possible.
• Analytics review. Review the metrics. If formative research and an analytics plan have been ongoing from the start. This is important for an activity to evaluate product health, measure product success, and tell the narrative of the product development both internally and within the client’s organization.