Measuring the results of your product operations projects is tricky since this role is still emerging. We asked our experts if there are any metrics that product ops should focus on to measure success.

Here is what they said:

Challenging

“Right now we don’t have any set measures in place as the work we do is mostly in the complex/emergent problem space. We have no hard and fast projects or areas of focus which makes it more challenging to apply measures upfront.”

Chris Compston, Product Operations Principal at Farfetch

Project by project

“We don’t have so-called ‘product ops metrics’. Instead, we think of every project as its own standalone product. This means that we measure on a project rather than function basis. For everything we do, we first ask ourselves: ‘What are we trying to achieve with this? What does ‘good’ look like? How do we know we’ve achieved it?’
“Usage metrics and qualitative feedback are our standards, but we also focus on whether and how much we’ve been able to establish connections between peers.”

Antonia Landi, Senior Product Manager in Product Operations at AVIV Group

3 key metrics

“Of the medium/high customer impact releases that have launched, how many were POPs involved in, and of those, how many delivered on time? If there were delays, were the reasons attributed to anything product ops could have prevented? For example, delays due to attrition or a change in strategic priorities is out of our control.
“Based on our stakeholder’s experience working with product operations on product launches, how satisfied are they with our launch management processes and the support we provide? How is the satisfaction score trending quarter after quarter? It’s important to understand where we can improve (e.g. tools, strategic guidance, processes) so we can make a plan to iterate and drive even more value.
“What is the overall sentiment to new frameworks/processes we roll out? When it comes to the programs we run, to be truly successful, I’ve found that being ultra thoughtful about change management is key and we want processes driven by our leaders, not the other way around. They know what is going to work and what isn’t - they are the receivers of our initiatives and if they don’t build alongside you, it’s likely there is a ton of context you’re going to miss.”

Tina Laungani Stewart, Director of Product Operations at Twilio

10 dimensions

“At this time we are focusing on a couple of areas. One, we created a list of 10 dimensions that the product managers were scored on and product ops will focus on initiatives that will help increase the PMs’ score on those dimensions. Two, we are also using metrics on completion and adoption rate of training or new processes that we implement to measure our success.”

Camila Gargantini, Senior Product Operations Manager at Oyster®

Psst... Looking for more insights into product operations? Take a deep dive with our Product Ops ebook. 👇

The product operations ebook: A comprehensive guide to the product ops function
The ultimate guide to Product Operations is here! Read on to learn about the fundamentals of product ops, the responsibilities of a product ops manager, and much, much more…
Product-Led Alliance | Product-Led GrowthRebecca Stewart

Team health

“Currently, the only metric we are measuring is team health, especially relating to the changes implemented. Measuring both their understanding, level of security, and role within the team.
“In the future we will be measuring more, looking into metrics around hiring, on-boarding, engagement, growth and retention. We’ll also be looking at parameters like throughput, cycle times, experimentation, quality and much more to understand how efficient we are in solving customer problems and delivering quality products.”

Hugo Froes, Product Operations Lead at OLX Motors

Satisfaction and lead time

“We focus on organizational and satisfaction, which is essentially a survey that we send out every six months that we had a baseline for before the product ops team was even formed, and we’ve been tracking this over time. It maps to our key focus areas that product ops is focused on.
“What that allows us to do is see the progress we’ve made over time. Most importantly, it’s a great way to collect qualitative feedback from folks who might not necessarily feel open to sharing them when we speak to them individually.
“The other thing that we’re starting to focus on now is lead time. Essentially since we’ve put a lot of effort into defining a launch process, thinking about how long it takes to go from the initial customer discovery call to post-launch and really trying to cut that down. The quicker that cycle becomes, the faster you learn and the quicker you’re able to ship. And shipping velocity is also something we track and we believe this is one of the most important markers of a healthy product and engineering team. Those would be some of the metrics that we look at right now.
“Whenever we launch an initiative, I push my product ops managers to think through success criteria for each of the initiatives they’re launching. This is the first quarter that we’ve become really outcome-oriented. I’ll have each of my product ops managers write out the outcomes they want to drive by the end of the quarter. For example, we’re just looking at closing the feedback loop with customers, which is something that we’ve struggled with in the past. Our product ops manager who’s leading that initiative has set a goal to close that feedback loop with 100 customers just this next quarter. Those kinds of initiatives and outcomes are tracked weekly now.”

Shintaro Matsui, Head of Product Operations at Amplitude

Surveys

“We do several things. First is an annual collaboration survey. We’ve seen over the years that the score the product management organization gets has been decreasing and decreasing. It’s just becoming harder to work within product management. And therefore, one of the success measures that I have is to increase that score year over year. I’m not saying that I’m going to take them from 20 to 50 in one quarter or one year for example. But I can start to prioritize my work based on that feedback because we’ll see an immediate outcome or return on that investment.
“The other one is every six months I launch what I call the product manager time allocation survey, which was really for me to understand where the 20 product managers that, at the time I was working with when I first started, were compared to now. Where were they spending their time?
“That was another data source that helped me prioritize my roadmap. For example, they were spending way too much time on support tickets and they weren’t spending any time doing market research. So, I asked what are your top productivity killers, and where do you want to spend more time? Are you currently spending time in the right places?
“The last one, which is probably the most important according to my Vice President, is our delivery to our customers. That’s one of his OKRs and it’s a lot more tactical, because that is the lagging indicator of what we’re doing early on in the process, to make sure we’re achieving the commitments to our customers.
“Because if we look at 2021 for the average delivery of our commitments to customers, we were sitting at 74%, which wasn’t awful, but it wasn’t great. Now that we’ve closed out Q1, we’ve reached 90% of our commitments. So we have the next three quarters to bump that up to 100. It’s all about the activities that are leading up to planning, and then refinements of the estimation. So it’s all those activities, all that process documentation, the access to data that will drive that improvement.”

Diana Soler, Senior Product Operations Manager at AppDirect