Kara Gillis, Senior Director of Product Management and Observability at Splunk, gave this presentation at the Product-Led Summit in San Francisco in 2023.

If your company has been around for longer than 10 years, I imagine you have some tech debt. There's a certain lack of telemetry available to you and some amount of guesswork that goes into your prioritization.

There are probably blind spots when it comes to the customers that you're talking to. Are you even optimizing for the right ones that make you the most money? Are there areas that you want to go and target more? 

A lot of customers that I’ve talked to are on the same journey that Splunk is: a 20-year-old company that's trying to get into product-led growth or has gotten into product-led growth, for example, in our observability business through acquisition or innovation, but they have a lot of stuff that they also have to take care of to modernize when they get there. 

If you’re like ancestry.com, then you're born in the cloud, and you're able to run all these experiments, but not everybody can do that right away. The goal is to get there. But a lot of these internal platform teams are trying to modernize and go to the cloud.

I'm going to talk a little bit about both the older world, the enterprise sales version, and also these digital native companies that I focus on at Splunk. 

Reaching your prioritized backlog

I've been at Splunk for almost eight years, and I've always worked on the observability side. I’ve seen this movement from 2016 to now, where we’re very cloud-first and almost cloud-native in a lot of our product areas, but we also have a legacy business that we're trying to manage. 

We have a unified backlog in our organization across both sides of the house, so we have to prioritize all the stuff we can do in the cloud-native area in addition to what we have to do to modernize our business. We have to make those two things work together. It's a very complex set of priorities that we have to think about.

Ultimately, we care about getting to that prioritized backlog and then putting it into a nice, pretty roadmap.

Depending on the time horizon that your leadership cares about, we often do it by quarter over a year, and we do a rolling roadmap, so we're always showing a year out. 

Now, how do you get there? 

Well, I think of a prioritized backlog like Google Maps. I have finite time; I need to reach a desired destination, and I have to put my criteria in where I have to go and my preferred method of transport.

What are the areas that I have to navigate to? Do I want to avoid tolls? Do I want to only take back roads? Or do I want to get there as fast as possible? 

A stack-ranked list of steps comes out of Google Maps, and that's basically a backlog. I need to have all these considerations about where I am in my journey as a product-led growth organization or what the status of my product even is. Is it cloud-native? Is it not? Is it a refactored monolith that's trying to adapt microservices? Where am I on that journey, and how can I get to where I need to go? 

Today, we're going to talk about: 

Understanding your customer

Let's begin by selecting our starting point. Where am I putting the starting line in Google Maps?

If I define my key objectives and results, I'm trying to understand who I’m targeting. We often start with the name of a persona at my company, but what does that actually mean? What problems am I solving for that person? That person probably encounters a million types of problems, but what is it that I want to solve for them? And how do I want to solve that problem for them? 

I’ve often found that the problem I thought I was solving wasn’t the right problem. So, do I understand the needs of the user in the context that they have? I don't want to be too general. I need to scope down so that I solve the problem that they truly have, and that comes from understanding as much feedback from the customer as possible. 

Also, you might not have a lot of feedback on the product itself, so how do you get that? 

I want to segment and then target a select user type. I want to get as specific as possible when I'm looking at a backlog. Sometimes, that's a pretty broad group of people, but I need to narrow it down as much as I can. So, what segments within that user type am I trying to target? Do I understand their uniqueness? 

And finally, how do I prioritize those needs? What’s the highest impact I'm going to make and the highest frequency of issues that I want to solve that’ll make my product as sticky as possible? And how feasible is it to solve that issue? 

It starts with active listening 

For the customer journey, you start with activation and onboarding, and then you get to retention and expansion. But from a customer's perspective, what does that mean?

In my opinion, activation and onboarding is day zero, and this is also the most important. If they can't start using your product, you're out of luck. You don't get a second shot. That should be the most important thing for net new customers, but also whenever they want to start a new use case. They’ll get frustrated and leave your product so quickly if they can’t get started with it straight away.

From a PM perspective, how do I reach my target to want them to try the product? That's part of day zero as well. Targeting them and getting them to that landing page and trial is just as important as the ability to sign up for that trial.

How do I also get them to the right place after they've signed up? How do I get them to the first step in the use case that they're trying to execute on? 

You’ve got to keep those users. They're going to leave you if you don't figure out how to keep the product sticky. So, how can I complete my first use case? I need to have an end-to-end feature and an end-to-end workflow. 

How do I unblock the barriers as a PM within that workflow? Can I repeat my use case? 

If I'm Dropbox, for example, and I want to upload and share a file, can I do that multiple times? 

Is my service reliable, secure, and scalable? Can I have many documents? Can I share it with many users? 

Is my data going to be hacked? Is it going to be sent to the wrong person who can access it even though I haven't given them access to it? The trust that your users have in your service is just as important as the features themselves, and I also consider those as elements of innovation. 

And then finally, can I complete a second use case? Often, we think about what features we can add next. This can also be, can I monetize or upsell them to a feature that they care a lot about? That expansion is incredibly important. 

Diagram of the customer journey - from activation & onboarding, to retention, to expansion

Differences in growth enablers offer different ways to listen

If I'm an enterprise product and I don't necessarily have access to a lot of telemetry, I consider that to be enterprise sales-led growth. Maybe you're trying to shift to product-led because salespeople are expensive. They're highly effective when they're good, but that's the old way. We want to get to the new way, but there's often a journey there. 

So, what are some of the ways that I can collect telemetry or get feedback from my users before I have access to complete product-collected data? 

I currently rely heavily on go-to-market teams for parts of the business. We have user feedback loops, customer advisory boards, and field advisory boards, meaning salespeople and technical field sales engineers will sometimes be proxies for customers because they're on-site daily. Splunk is a very technical product. It's very important to the business, but it’s also very complex. 

So, how do I get as much data as possible and as often as possible if I don't have access? Maybe there's a black box in there. Maybe it's an on-premises product. Maybe they have a firewall that I can't access because they haven't signed up for the cloud service yet and they're migrating to cloud, so before they opt into my cloud service where I have access to everything, I have a black box.

I need to figure out how to get that information, and then I need to instrument that information when I have the cloud service, when I have all of the data I need, I see all of the journeys that they're taking, I've adopted microservices, and I have all the telemetry instrumented.

It's very important then that I have in-product analytics and usage data, and I'm running experiments constantly. 

The elements of a backlog

What are the non-negotiables in a backlog? 

It makes me angry when a service doesn’t prioritize security, reliability, and compliance from the beginning, especially if I'm using some sort of telehealth.

A good example would be BetterHelp. You hear about it on podcasts all the time. They're talking about all of these therapy portals that you can go in, and it's so much cheaper and so much more attainable for people because therapy can be very expensive. 

But what if you log into that portal and it's not a secure portal? Those portals often require you to write your deepest, darkest secrets into it. 

They say that they're HIPAA compliant and that they're secure, but what if they're not? All of your information could just end up on Reddit.

That's not okay. For me, these are the non-negotiables. They build user trust, and they're also part of innovation and finding ways to provide a secure and compliant environment, especially if you're in a highly regulated industry like financial services. 

Also, if I can't transact or add to cart and I have a bug fix that I need to look at, that comes before everything else. 

Now it’s time for the negotiables. This is where we have new features, feature enhancements, and tech debt. Tech debt isn’t fun, but it’s so necessary. 

Tech debt isn’t a non-negotiable for me because there’s more decision-making power on when, where, and how long it'll take you to fix it. 

If you’re on that journey of trying to get to product-led growth and trying to get to a cloud service, you have more leeway here than you do on the security, reliability, and compliance issues that your regulated industry might be facing. 

Prioritization can look like a spectrum

Now, let's talk about not only the type of industry you're in but also where you’re at in the product lifecycle. 

It matters where you're going to prioritize certain things. If it's a new product launch and you're a brand new startup trying to figure out how to find product-market fit, and Walmart or McDonald's comes to you and says, “Hey, we'll buy your product if you do that certain thing for us in the product that we really want.” 

Maybe you've already come out with a product-market fit, and maybe you even have a couple of products and you know where you are in your journey. You're trying to expand. Splunk started in IT ops, expanded into security, and now we've expanded into more of a developer community, so we're very established, 

What I’d consider important might be very different from where you are in your journey. 

Diagram demonstrating how prioritization can be a spectrum, from a new product launch to a legacy product

This is a scale I like to think about from a new product launch perspective. You need to make sure adoption is number one, but with how many customers you already have and how long they've been your customers, retention starts to take shape. You need to keep your customers. 

ARR calculations like recurring revenue come from your existing install base, and then you can start thinking about higher levels of monetization and other areas of use cases that you want to add to the product. 

But you have to focus on where you live today. You have to attract the customers and keep them using your product before you can start adding other things willy-nilly. Nail the things that you need to be good at before you start to add other things. 

I see a lot of companies start to get very unfocused, and they try to do too much too soon. Really nail the adoption path, then nail the retention path, and then finally, you can expand. 

Once you start having all these customers, you’ve got to think about scale. Are thousands or millions of users using your product at the same time? 

A very good example was the Ticketmaster debacle with Taylor Swift. It was very frustrating for many people not being able to transact, and the entire thing crashed, leaving a very bad taste in their mouths.

Scale is super important when you've reached a certain level of product-market fit. It's often overlooked, and that's part of the tech debt. Was there enough infrastructure capacity? Was there enough CPU to drive that much transaction volume within a minute of putting Taylor Swift tickets online?

And now you see that Taylor has even bypassed studios and gone straight to AMC for distribution of her Eras Tour movie because she no longer trusts the distribution mechanisms that have been in place. 

So user trust is incredibly important. A distribution company has missed out on that opportunity to publish Taylor’s Eras Tour. She’s cutting the middleman out because she’s lost trust in a service that she relied on for her own business. You don't want to be left there. 

I believe that for tech debt, you should always budget 10-15% of every quarter to address it. 

We're trying to get as many cool new features out, to be competitive, and to have our customers excited about using our products, but we don't address some of the tech debt. It's like a credit card bill that you haven't paid. It will be due, and if you don’t address it in a regular fashion, it’ll bite you in the behind. 

So that's why I always like to have a little bit every quarter that I'm addressing. I try to get the biggest, knottiest things off that list first, and then I can start to go down the list and it just becomes routine and it never overwhelms the roadmap. 

If tech debt starts to become the biggest issue in your roadmap, all of the innovation budget goes out the window. And then you're basically just a tech debt shop for a couple of quarters, and that can demoralize not only your engineers but also your customers won’t have new features delivered to them. So it's always better to have a little bit of tech debt addressed every quarter or every sprint.

Adding new features

Features are fun, and enhancements to new features are even more fun because you can make them even better. 

Adoption and onboarding new features is when you prioritize efforts to have a seamless day-zero experience. I can log in, I can see something that makes sense to me, I can select a workflow, and then I can complete that workflow. 

Retention is when people don't lose interest in the product. If they’ve not been able to complete the day zero, then they're gone. But let's say they're able to do one use case but then they're not able to do it again. How do I fix that? 

Or if they tried to scale up too quickly and got booted out, what are you going to do about that? 

Let's say you're watching Love is Blind on Netflix and they’re having real issues with the streaming service for a live taping of the reunion. In the end, it doesn't come on for 22 minutes. You get frustrated with Netflix, and then you lose interest in the episode. You have a lot of drop-offs, and that becomes problematic. 

Finally, the expansion and monetization. This is also very fun because you have a lot of data by this point, you've reached some product-market fit, you have people using your service, and you’re ready to enhance that product and have people pay for it. 

But how do you do that? 

Well, this is where the in-product usage and telemetry becomes really important. 

The three steps to follow when adding new features: step 1 - adoption and onboarding, step 2 - retention, step 3 - expansion and monetization

Refining your priorities

If I have a prioritized backlog, in my opinion, reliability, security, and compliance come first, then bug fixes, then the new features, then enhancements to those new features, and then tech debt at the very end. 

And then I can prioritize them based on the quarters that I want to deliver them in. 

There are many ways to prioritize how you select which month or quarter to deliver them in, such as the RICE framework and the Kano framework, which is about basic performance and delightful features. You can also use the Eisenhower Matrix, which talks about the urgency versus the importance of features.

Now, if I want to refine the backlog, let's say I have scale issues like that Ticketmaster example. I’d make sure that enhancements come before new features, and I’d put tech debt above reliability, security, and compliance. 

Secondly, let's say there's a brand new digital native company that’s threatening my existence; the number one priority becomes the new features. 

What if I have something coming from other parts of the business? You have to expect there to be a surprise in every quarter. 

So if you budget a little bit for that but keep your list very steady, where does it fall? It's not going to overtake the entire roadmap if you have a very specific set of criteria that matters to you. And you can slot it in and budget whatever that line item is into the prioritized list that you've already created.