Hi, my name is Lucy Huang, and I'm a Product Manager at FullStory.

I've spent a lot of my career thinking about all the things that can go wrong. I currently work in the trust and privacy space.

I've also worked in an incubator, where I tested out new product ideas every single quarter, trying to find product market fit. And that's why I'm here today, to talk to you about de-risking your product ideas. And I'll also be sharing strategies and tactics to get to delivery faster.

The consequences of not addressing risk

First, let's define risk. Specifically today, I'm going to be talking about how you can de-risk that time spent to do discovery around a problem, and also how to do discovery around a solution so that you can get to delivery faster.

We're going to figure out how can we make sure that we’re addressing that problem, addressing the problem at the right time, at the right place, and for the right person.

But first off, what are the consequences of not addressing that risk?

Well, first off, it can be a cost to your business, including head counts and your budget. It can also be a cost to your competitors. Where are you going to be losing to them or lagging behind?

It's also important to take timeliness into account in this economy, and make sure that you're advocating for the time and resources to be able to do discovery properly. That way, you don't jump into the solution too fast.

But before you go looking for too much risk, it's important to understand that as humans we probably look for problems too much rather than preventing the problem in the first place.

Do you have trouble remembering what you did last week or last month and what your accomplishments were? That's because as humans, we’re naturally inclined to shift our focus to the next problem. And it's hard to pause and remember all the accomplishments that we've actually done because often our job as product people is to move forward and unblock or execute whatever our team needs to be done.

And that's why there's often this issue of a preventable problem paradox, where that problem prevention doesn't feel very visible and it feels boring.

In contrast, problem-solving's very visible. You're critical to your organization. You're the savior, you're the hero. But that's not sustainable in the long term, and that's why as an organization, you need to foster that culture of problem prevention.

It's really important to develop a consistent culture of problem prevention before you go looking for too much risk and all the things that can go wrong.

Top frameworks for risk

Next, I'll jump into frameworks for risk, understanding what types of risk need to be addressed and when, and who the directly responsible individuals are for owning the outcome of those risks.

The SVPG principles of risk

Silicon Valley Product Group’s/Marty Cagan’s principles of risks are classics for a reason. And to effectively execute on addressing these risks, you also need to address some of them early on, especially that business and value risk to make sure that you're setting your teams to work on the right items.

You'll also need to recognise owners and assign those directly responsible individuals to make sure that you're owning those outcomes.

It's also important that you work collaboratively. What I've seen a lot in my time is that you often have product and user experience folks owning that usability and value risk. And by the time it makes it down to engineering, you realize that this isn't actually possible. So that's why you should be working collaboratively as much as possible, rather than that waterfall mode.

So we'll go through each of these risks:

Value: Will your customers buy it or will people choose to use it? That's going to be something that's owned by you, the product manager.

Usability: Can your users figure out how to use it? Can they figure out how to navigate it? That's going to be owned by your design or user experience folks.

Feasibility: Is it possible to build this with the times, tools, and technologies? This is going to be owned by your engineering team.

Business viability: Is this going to work for the other various aspects of your business? Is it a compliant solution in the first place? Should you be focusing on this item to open one revenue stream, when you could actually be focusing on unlocking two to three revenue streams with one initiative? It's really important to make sure that you're addressing these.

How to prioritize risks

Now that you've figured out the different types of risk, it's also important to prioritize your risks. I have these 2x2 matrices below. I love these; they’re a really great way to frame things.

So we have two scales, your critical and non critical. So this is basically figuring out how important that risk is and how critical or non critical it is.

Then we also have known and unknown risks, figuring out how much information you have about the risk. So we'll walk through the quadrants here.

First, that evaluate portion. This is something that's important and unknown. So you're not ready to continue here without more information, and you probably need to consult some domain experts.

For your unknown and non critical portions, that's where you need to do generative research. This is something that I always advocate for as a consistent practice in product organizations.

If you're familiar with ‘know thy user’ or ‘know thy customer’ interviews, they’re basically weekly or bi-weekly interviews that are often driven by the product manager or the user experience team to make sure that you're understanding your product broadly as a whole, and consistently generating new ideas.

So that way, as you're developing features, you're not too zoomed in. You're still able to understand as a whole how your product’s doing and prevent product drift over time.

If something's known and non critical, then feel free to punt it with gusto. We all have enough on our plates to tackle later on, if at all.

And if something is known and critical, you can probably plan ahead here. For example, if you know that you need penetration testing to prove how secure your product is, you can probably go ahead and move forward with that.

I’ll also talk about another way to prioritize risk. And this is something from a guy called Jeff Bezos. You probably know him. He has this great framework for Type 1 versus Type 2 decisions.

Type 1 decisions are decisions that are consequential and nearly irreversible. They're basically one-way doors that once you cross to the other side, it's really hard to go back. Examples of these might be developing a data model that all your analytics will be based on, designing your API, or maybe choosing a vendor for your cloud infrastructure. It's going to be really hard to undo the consequences of that decision.

But what actually ends up happening most of the time is that you have Type 2 decisions, where they're actually changeable or reversible. They're basically two-way doors, where you don't actually have to live with the consequences of your decisions. If it’s a suboptimal one for that long, you can go back.

As a PM, or as product folks, it's really important that you own the speed of that decision-making. You might not need to be the owner or the one that makes the decision, given subject domain expertise, but you are responsible for that speed of decision-making.

Key methods to de-risk

Now we'll talk about how you might actually de-risk during discovery, as well as during the problem phase and solution phase.

De-risking the customer problem

Below is an overview of how we might direct the customer problem. Surprisingly, or maybe not, a lot of product teams don't actually follow these steps. I think it's because it's a really hard discipline to follow.

Let's say that you're diving into a new space and you've been tasked with understanding why user retention is down. Now, this isn't an interview question. And unlike an interview question, you need to be able to advocate for more than 10 minutes to spend time during proper discovery and make sure that you're solving the problem for the right person at the right time and the right place.

So, let’s jump deeper into this process.

Establish your assumptions

First off, you want to establish your assumptions. What are those leap-of-faith assumptions? What are your first principles in this case? This exercise will really help you identify those hypotheses that you want to test and be able to test them throughout research.

Questions you might ask are things like, “Do you have a persona in mind? Do you know what their jobs to be done are? What are the workflows or steps that you think your users take? Can you map out these workflows and take them to the user later on?”

Also ask them, “Hey, do these steps actually reflect what you do in your day-to-day?”

It can be a very collaborative process, where you and the user are whiteboarding together on what their process actually looks like.

At this point, you don't even want to go into solutioning or building out a UI. We're really focused on understanding the user's workflow and where their pain points are.

Talk to your customers

And again, back to the basics. Talk to your customers. Understand where they’re feeling the most pain, and when and why they’re feeling that pain.

Tactical ways to do this are jumping on sales calls and listening to pre-recorded calls if you have access to tools like Gong.

When I worked in an incubator, I used to ping random people on LinkedIn all the time and ask them for 30 minutes. That way, I could understand where the biggest pain points were that they were running into.

Then I’d go and build out a custom solution or prototype for them, leverage that network later on, and say, “Hey, I've built a solution for the problem that you've told me about. Would you like to try it out?” All of them said yes, because they knew that I understood their pain points. And whatever solution I built probably worked well enough for them.

User research methods

There's also a whole landscape of user research methods that you can take. And you'll probably want to do a mixed methods approach, using both qualitative and quantitative.

But one thing that I find is missing from a lot of frameworks like these is something that's rather pragmatic, in my opinion, which is ‘time.’ That's often what we're running against most of the time.

So consider that trade-off: one month of user research and discovery versus six months of actual development. Of those, the lesser evil is probably advocating, again, for that one month, to do discovery.

But a lot of these methods can be hard to fit into that one month. So it's important to move at speed and make sure that you're winning against your competitors, and also supporting your customers.

Below is another two by two matrix. Here, I've plotted out those qualitative and quantitative user research methods against those resources. Here, we're maybe talking about time, or maybe the number of headcount that you have.

I’ve found that it's often one to two user experience research folks for every 10 to 20 product managers and designers. And often, what this leads to is a more consultative relationship.

You might get the random one-off where your UXR is embedded into a project and driving most of the research there. But often more than not, it's a consultative resource. And because of that, a lot of these user research methods can require a lot of resources.

For example, field studies and diary studies can take a lot of time to set up, gather, and synthesize information. Even A/B testing can be really difficult because you have to take time to set up that infrastructure.

So if you need to move fast in terms of de-risking discovery, these two things will get you there: user interviews and usability testing.

Now we'll go on to that ideation. What have you learned from your generative research?

If you've done those ‘know thy customer’ or ‘know thy user interviews’ where you're constantly understanding where your customers are running into some of those problems, you're probably ready to jump into the ideation phase.

Opportunity solution trees are a great tool here. I'd also make sure that you're advocating for mapping out the crawl, walk, run scenarios of those opportunity solution trees and mapping them out to the actual impact. More often than not, we get tied to our grand vision, but it’ll take two years to execute. So that's why it's important to map out those crawl, walk, run phases as a part of your solution tree.

You can also use learnings from your generative research to drive some of the ideas and inform your hypotheses of how we might address these customer problems.

De-risking delivery

We've talked a lot about problem discovery. Now, we'll talk about solution discovery or that delivery portion and how to de-risk there.

Shreyas Doshi is another great product leader. He talks about pre-mortems. That's your action item from today. Again, we're running into that preventable problem paradox.

For me, I think of pre-mortems as almost like therapy. As a product manager, I tend to catastrophize things and worry about all the things that can go wrong. I especially run into a lot of these concerns with my engineering team, and that's because any complex organization will probably tend to incentivize problem-solving rather than problem prevention. So don't go looking for problems and creating artificial barriers.

For example, let's say you have a bunch of support tickets coming in that are always around the same issue. Rather than addressing the core of that problem, you might have a process that says, ‘If this bug or issue ever comes up, whoever's on call will address it.’ And that person who’s on call is painted as the savior or the hero. They're critical to your organization for fixing that bug.

That type of problem-solving feels very visible and very critical. But what you actually need is problem prevention, something that could’ve prevented this from being an issue in the first place.

But that can often feel boring. It's hard to commit resources to it. And oftentimes, most people just want to be able to respond in the moment.

So here, pre-mortems can be your takeaway. They can help set expectations, and again, foster a culture within your organization of preventing problems. This’ll also reduce your chances of a tough post-mortem later on.

For your pre-mortem, you'll want to figure out and understand all the things that go wrong and what led to this. You'll also want to prioritize risks that you've evaluated. Again, you'll want to figure out how important these risks are to address. Which ones will you start to plan for? Which ones will you start to evaluate? Which ones can you punt for later?

You'll also want to assign directly responsible individuals for each of the areas that we’ve talked about, such as that usability. if it's a usability issue, your designer probably needs to take point on that. If it's a question of feasibility, your engineering team probably needs to take point on that.

When is it right to use RITE?

Now, we're getting into my favorite part of the presentation, which is more of an evaluative research phase. If you need to move fast with solution delivery, I'd highly recommend the RITE method. It stands for rapid iterative testing and evaluation. It's basically a way for you to leverage user interviews and usability testing within a very quick, rapid framework.

The goals of this are to be able to get feedback quickly and iterate quickly as well.

So rather than your typical UXR rounds, where let's say you have 10+ participants each, you have to set up the calls, you have to set up the questionnaires, put together notes, synthesize them, and share the findings of your team, I'd actually see where you could be more lean and move forward with maybe two to three rounds of participants each.

So what you'll do is put together an initial prototype or design. You can show that to two participants, highlight any major usability issues, update the design, and then run it by three participants again. If there are more issues, then you can go ahead and update the design again as well.

What this works really well for is being able to move quickly. And again, you have two to three participants each. But it's also important to understand that if it's a low-fidelity prototype, such as something that you have in Figma, that can be done by your designer, or if it's a coded prototype that your engineering team has to own.

If it's a coded prototype, then it might swing more towards iterative testing and rapid testing. So it’d be ITE instead of RITE.

It's also dependent on your timeframe. This is going to be a very intensive process if you do come down this way. And it's because your designer and engineering team need to be with you for each of these interviews.

I invited my designer and engineering team to each of these interviews because the expectation is that you only need to go to two participant calls for each sprint. And because of that, we can get feedback from the users right away and ask questions directly instead of back and forth.

And instead of having to focus on taking notes, synthesizing information, and collecting it, right away after that call, my user experience team, designer, engineering team, and I can go straight to storming and forming what improvements we want to make, rather than that long funnel of back and forth and context sharing. You can go straight to product development instead.

Make sure that you're adapting a schedule to your prototype. Understand if it's going to require an engineering team to lead the development of that prototype, and also make sure that you're sharing that schedule with your team. It can be very time intensive.

After two rounds, my team and I were like, “We're done. This is a lot. This is great. We have enough improvements to keep us busy for the next month.” And it's because you can get a wealth of information from studies like these that are conducted in this way.

It's also important to make sure that you're keeping a decision log if you're moving at this speed and pace, and understanding why you made decisions to prioritise certain items over others. Keep snapshots and version history names of what you're focusing on as well.

Key takeaways

So we talked about understanding the different types of risk: the value, usability, feasibility, and business viability.

We’ve talked about Type 1 and Type 2 risks, where often, most decisions you have to make are actually Type 2 decisions.

We've also talked about how you might prioritize risks, and whether you might plan, punt, evaluate, or generate more ideas through something like ‘know thy user’ or ‘know thy customer’ interviews.

We've also talked about specific strategies to de-risk, such as using pre-mortems to foster a culture of problem prevention.

And then we've also talked about the RITE method, ways to quickly conduct evaluative research during solution discovery.