Product ops evolved: lessons learned from scaling a team from 1-5 in 18 months

This article was taken from a presentation by Shintaro Matsui at the Product Ops Summit. Catch up on all presentations with the OnDemand feature.

My name is Shintaro, I head up the product ops analytics team at Amplitude. We're a product analytics company that helps companies build better products. It's been really exciting to see the momentum product ops has had in the last year and now more than ever, I have the conviction of just how critical product ops is as a function, not only for product development teams but for companies as a whole.

“The people in the systems that create the product are as much the product as the product itself” is something John Cutler said to me early on in my journey which has stuck and resonated with me. I really couldn't agree more.

If the process to launch a new product is subpar or if you have leakages in collecting customer feedback, it makes it challenging to build and ship a compelling product that adds value to customers.

Instead of going deep into one topic, we're going to cover a wide-ranging set of topics, everything from selling the product opposition to how you ask for more resources, to launches, and frameworks.

Here’s a breakdown of the main talking points:

• My product ops journey
• Team of 1: This is fine…
• Then there were two
• Team of N: A whole new set of challenges
• 🔮 What’s next?

My product ops journey

I joined Amplitude a year and a half ago coming from an operations role at Uber and having really little idea of what the function was, did or looked like at a B2B SaaS company.

A lot of it for me was figuring it out on the fly and I faced a lot of challenges as I scaled the team. So hopefully, you all can get ahead of those, and not repeat the same mistakes that I did as I share specific tactics I've used over the last 18 months. I want to get super specific in terms of the tactics and hopefully, you'll be able to implement a couple of those tactics for your own team.

I've structured this article in three parts, which represent the three stages of my journey within the product ops discipline. At each phase, I found that you'll come up against different challenges that are quite distinct. The first stage was when I was a team of one.

I would say that is a really challenging role to be in and have tons of empathy for that particular role. Next are the challenges that I faced as a team of two. The third chapter will focus on a team of N, or today at Amplitude, a team of five.

Team of 1: This is fine…

When I joined Amplitude, I was a team of one. The image below, is really how I felt when I was a team of one at Amplitude. You join, everything's on fire, you feel this almost moral obligation to fix everything, as soon as possible and it really feels overwhelming.

Challenges

🧭 Defining the vision: Unpacking what product ops mean for your particular company.

💰 Proving value: Thinking about proving value is something you want to start to do right away to show how product ops can add value for your particular organization.

⚙️ Doing the work!: The challenge is actually doing the work, not only do you have to define the vision and the strategy but you (as a team of one) now have to execute against that. How do you go about doing that in a sustainable way?

Tactical advice for defining vision

Discovery sprint

In terms of defining the vision, one piece of tactical advice I had and was helpful for me was to have my two weeks into the role be a discovery sprint.

I structured that sprint in three key steps:

1. One-on-one’s → I set up about 35 one-on-one’s in my first week at the company, with various teams to understand where their pain points were. In a lot of cases, I felt like a therapist but it was great because folks felt like they were able to share some of the pain points they were feeling on a day-to-day basis.
   
2. Pain points by teams → After I had those one-on-ones, I asked everyone if I could fix one or two things for them, and what would be the highest impact items for product ops to work on. I took those pain points and then started to cluster them by team.
   
3. Impact vs. effort → Lastly, I would map those out into a two by two based on impact and effort. This allows you to ground yourself in the pain of the organization. I see a lot of new product managers have issues when they come into an organization, they create a framework they've seen work well at other companies and actually try to apply it full stop across the organization. The issue with this is it doesn't solve the pain that an organization is feeling and it's important to ground yourself in the pain of the organization.

Maturity model

The second piece of advice that helped me as I was defining the vision, was to come up with a maturity model like the one below. Essentially, what it helps do is ground your organization and where you are today, but also where you want to be in the longer term.

When I joined, I explained to my team that we were at a foundational level of product ops maturity, we were building these foundational tools and processes that should have been in place years ago.

We then moved on to an operational level of maturity where product ops is playing a critical role in the day-to-day operations of the company, through launches, or through owning all of our product metrics, for example.

Finally, the vision I like to set out for the team, to get people excited about product operations, is that it can really be a competitive advantage. When all product-related processes can be automated or optimized such that any team could work seamlessly through them, it allows those teams to get a lot of time back to work on high leverage activities like speaking to more customers or coaching their team. A maturity model was helpful in sharing that longer-term vision.

👎 Failure

Reflecting on failures, when I was a team of one, one thing that particularly came to mind was wanting to work on the highest impact piece of work regardless of effort.

When I joined and spoke to everyone, they told me launches were a mess. I thought, “Great, I'm going to create a new launch framework and then solve all of these highest impact issues”. I quickly went down that path and started to align a lot of stakeholders, but quickly realized that for week two, I had bitten off way more than I could chew.

Then I quickly pivoted and realized it's much better to chalk up some quick wins to build momentum for the product ops function, as opposed to these big bang efforts.

Tactical advice for proving value

Quick wins

A great way to prove value is to chalk up some of these quick wins. An example of this was PDN 60. The problem we were trying to solve is, in speaking to go-to-market during my discovery sprint, a lot of folks were saying that they're often taken by surprise with launches.

What can we do to solve that? I spun up this newsletter which took about a day to put together and had product folks add updates for what they had worked on in the last month, shared it with the whole company, and people loved it.

What's interesting is now a year and a half later, this has served us well at the time, we've made a lot of investments around a documentation repository and a knowledge management system and we're asking ourselves if we even need this newsletter anymore. It's a lesson in not getting emotionally attached to processes or initiatives that you kick off. It's a success if you kill them off or if you iterate them because it shows that you're evolving as an organization and as a company.

Close the feedback loop

In a similar vein, when you're talking about tactical advice for proving value, I would encourage everyone to always close the feedback loop. What I mean by that is, when you're doing those discovery one-on-one questions and folks share specific pain points they have let them know you solved them.

Go back to them and say “Hey, I remember you had shared this really brutal pain point, here's an initiative that we've launched, is that meeting your needs” and make sure they're aware of the effort that's being put in.

This helps create trust and goodwill, which is important as you're building out allies for the product ops function but it can also help with resourcing. When you're making new resourcing requests to build out your team, you can have those folks who've seen the value that product ops create advocate on your behalf.

Tactical advice for doing the work

What I always encourage folks to do is to start really small. Let's say you want to launch that new launch framework we mentioned above, instead of feeling like you have to put pen to paper all on your own, build that framework with key inputs from your stakeholders and almost co-create it with them.

What's really helpful there is you have people from different perspectives (go-to-market, product, design) who are able to feel like they have a voice and feel ownership of that framework.

Then once you have a good v1 or even a v0, pilot it with one pod, a really small team. What that'll allow you to do is to be able to fail with a small blast radius, and then allow you to get quick learnings that you could then integrate into that framework and continue to improve it over time.

Once you've launched it with one pod, you could expand that to launches within a couple of pillars. Then by the time you have to roll it out to a whole product team, you'll not only have ironed out all of the kinks but you'll have a lot of PMs, designers and engineers who are going to be advocates and champions for this new framework because they've seen it and been a part of creating it from the start.

Summary

A recap for challenges and tactics as a team of one would be to use a two weeks sprint to help define the vision, prove value through quick wins and when you're a team of one that has to do all the work start small and incorporate other people and other cross-functional stakeholders scaling initiatives. If you do that, it'll help you justify moving on to the next stage, which is as a team of two.

Then there were two

When I made my first hire, I felt this sense of relief that I was no longer alone on this island of product operations and that I could scale my impact. You'll quickly realize now that you have someone to take care of and responsible for, and a whole new set of challenges arise.

Challenges:

🙅 Saying “No”: You've gotten that resource now and people expect you to have extra capacity to do even more work and to say no, is really important.

👥 Scaling the team: Now you have that team of one, how do you continue to scale that team going forward?

📥 Managing ad hoc requests: How do you go about managing ad hoc requests effectively?

👎 Failure

One key failure that I had when I was in a team of two was saying yes to everything. I wanted to demonstrate the value that our team could provide across a broad cross-section of pieces of work and this was detrimental for a few reasons beyond burnout. It gave false expectations of the function’s capacity, which is really hard to rightsize going forward.

Tactical advice for saying “no”

Draw the line

I made a slide for my manager, in which I distinctly outlined what our team could accomplish with the resources that we had. Everything above the line we do accomplish, with more resources we could accomplish the things below the line.

Being explicit with your team and your cross-functional partners help create alignment and foster the right expectations. It's also helpful to get people excited about everything that's below the line to help them look for the resources for your team.

If you're continuously saying yes and taking on new work, you're not able to close off initiatives you kicked off. What we like to say at Amplitude is, “Stop starting and start finishing”.

Tactical advice for scaling the team

Test the embedded model

How do you go about scaling a team? A helpful tactic at Amplitude was to test out the embedded model with my product ops team member. We embedded this product ops manager within a product pillar, which is a team that owns a large surface area within our product, and had that product ops manager attend all of their rituals.

I asked that manager to put together a list of all the observations that they had and improvements they can make from that function and had them support a launch. What happened was, people were soon able to see the value of having someone in that role. There was a lot of word of mouth that spread and other pillars soon wanted to have an embedded product manager as well, which was a helpful pull in asking for more resources.

Tactical advice for managing Ad Hoc requests

Sprint planning

There are always going to be pressing ad hoc requests, whether it's launches or m&a and a great tactic for managing those requests for the product ops team is doing sprint planning. This is sprint patent planning specifically for the product ops team, as opposed to the broader organization.

The benefits that this has had for our team are a fewfold; it allows us to track incoming requests, provides visibility to the rest of the org as to what we're working on and it creates empathy for other product teams who are running sprint planning themselves. We're able to share best practices and how we approach sprint planning.

Summary

A summary for teams is to have to draw the line to help delineate what you can commit to with what resources you have, improve your chances of scaling the team by piloting the embedded model, and then help manage those ad hoc requests through sprint planning.

Team of N: A whole new set of challenges

You might think, once you’re a team of N you've really made it, you have all of the resources that will really help solve some of those issues that you've had but that's not the case, you're gonna have a whole new set of problems as you face as a larger team.

Challenges:

🧱 Structuring the team: It's great, you have all these resources, but how do you organize them to be the most effective?

💡 Scale learnings: You're going to have each of these team members working on their respective workstreams and their respective product teams, how do you bring those learnings back to the organization and scale them?

🗄 Managing processes: You'll now be launching tons of new initiatives, how do you make sure they’re still being effective and you're revisiting them on a regular cadence?

Tactical advice for structuring the team

Embedded + majors and minors

The first challenge was structuring the team. The approach we've taken at Amplitude and what's worked well so far is a combination between an embedded model and product ops majors and minors.

The embedded model essentially means that each of our product ops managers are embedded within a particular product team. Why I really love this model is it helps product ops managers develop specific product expertise and it helps us develop deep empathy for those respective product teams.

For the PM, product ops managers have the pulse on everything, how go-to-market is feeling, and what customer feedback they're hearing so embedding them in the team allows them to become that strategic thought partner.

What I love about majors and minors is it helps product ops managers develop specific product ops competencies while also being able to dabble in other areas that they're interested in. For example, we have a product manager whose major is program management, but their minor is building and launching because that's something that they're interested in. One of the beauties of product operation is the breadth of work that can be taken advantage of and folks can be exposed to.

Tactical advice for scaling learnings

Frameworks

Next in terms of scaling learnings, I would encourage you to create your own company frameworks. Problem spaces are specific from company to company and while there are best practices out there, it's important to adapt those to what makes the most sense for your company.

Below is the product development framework we put together over the last quarter. Once you have this framework, we've realized it's helpful to codify these for your company and make sure that folks have a voice in building those out. It's also helpful to do a retro on these on a regular basis to ask yourself “hey, is this framework still serving us today? Where did it fall short?” and continue to improve over time.

Prod ops team leaning reviews

Another way we've been able to scale learnings is a key learning review. This is an hour-long meeting every month where folks on the team get to share what they've been working on and the learnings they've gleaned from the last month.

This is a great opportunity to bring in other product ops guest speakers. I've been guest speakers for other learning reviews and it's always great to chat with other product ops teams and to hear how they're approaching and what they're learning from their respective teams.

Prod ops office hours

Another way to scale learning is office hours, and you could have a couple of flavors of office hours. One is you have office hours after you've launched an initiative, so for example, product ops ran our annual product planning process and since it was a complex process, we offer 30 minutes of office hours every week where anyone can come by and provide questions or feedback.

The other flavor this could take is the more generic office hours for product operations where you can invite the whole company to come and share feedback and ask questions. This is also a great opportunity for product ops folks to present new initiatives that they’ve been working on as well.

👎 Failure

One big failure I've had when I was reflecting on my time as a team of N was assuming that a process that works well historically will continue to scale and there are many processes that I implemented a year and a half ago that just don't work anymore given we've almost doubled our team size today.

It's also important to think about deprecating or modifying the process as a success, it shows you're continuing to grow, you're continuing to evolve as an organization and it’s important to not become emotionally attached to them.

One way to think about it is as processes and systems as gardens that you continuously need to be taking care of and pruning regularly.

Tactical advice for process overhead

Process reviews

A tactical piece of advice around managing process overhead is our quarterly process reviews. The image below is the mural board that we ran to do this and all those large blue boxes are recurring processes.

We brought in a group of folks both on the leadership team but also more folks that are on the ground, like designers and PMs and we asked them to stack rank those processes on a scale of one to 10 based on what's most helpful and effective. It's fascinating to see how folks are feeling about processes you thought were working incredibly and are actually effective and vice versa.

Then, we asked folks to add specific suggestions for improvement. If you do this on a regular basis (quarterly or by the end of the year) you're going to have sound and tight processes.

Summary

In summary, when thinking about structuring the team, don't be afraid to pilot the embedded model with majors and minors, think through frameworks and learning reviews as a way to help scale learnings, and then use those quarterly process reviews to edit down and continuously improve your learnings.

🔮 What’s next?

What's next? How does the product ops function continue to evolve? I definitely don't have all the answers but I've seen a few interesting things in the market.

• Applying that operational knowledge to other problems based within the product world, like design and research and you're seeing those research and design ops functions emerge.
• Start to take on a cross-functional scope. Tim Chen, who worked at product ops at HelloFresh, in the UK, was explaining how the product ops team there owned a lot of the assortment planning and some traditionally DevOps functions because their team was so strong.
• One-to-one mapping between product ops managers and PMs, because they're seen as critical to the function today.

Common questions

Q: What is the biggest challenge you faced scaling a product ops team? How did you overcome this?

A: I think one of the key issues that I faced and came back to some of those discovery sessions that I talked about, which was as you scale, the team and your organization scaling, the pain points will evolve.

It's always important to ground yourself on the pain points of the organization. Make sure you're continuously going back and having that pulse on what's burdening the organization because what might've been burdening the organization at 300 people might be very different just six months later when you're a team of 500.

I would just say to constantly make sure you have a pulse and that you're solving the right problem.

Q: What are some of the key similarities and differences between site reliability engineering and product ops teams?

A: It depends on the function, at Amplitude we don't have a site reliability team, it's within the product ops team but there's no right or wrong answer. I encourage folks to chat with one another.

I've learned a lot in terms of how to structure my team or what other folks are doing on their teams and that's been incredibly helpful as I've tried to scale my learnings over time.

Q: Who do you include in your process reviews? Are they typically group reviews or one-on-one?

A: What we do is we do them in groups. We held my process review with the product leadership team and what's interesting is you see what's most important for that particular subset of stakeholders.

Then, I have folks on my team running similar sessions for product managers or designers or engineers and when you're able to see how those different groups of stakeholders view and value certain processes, you glean a lot more information than just doing it on a one-off basis.

Q: How do you find really good hires for product ops?

A: If you're scaling the team, the profile you look for probably depends on the scale of your team. When you're making your first product ops hire, the profile that was helpful for me to look for was more of a generalist because this person is not necessarily going to have specific domain expertise but they're going to be working on a tonne of different programs and projects.

Having someone who might be coming from an ops background or consulting, who's flexible in terms of the different kinds of work that they can lean into, was what I looked for in my first hire.

For my fourth, fifth and sixth hire there might be specific needs we're looking to fill like a research operations function, for example. I would really want to have someone who has good subject matter expertise in that domain, and who could really own that particular function.

Q: How do you measure the impact of the product ops teams, both in the planning phase and when reflecting on the changes you implemented?

A: How we think about it on the product ops team at Amplitude is we have a Northstar metric, which is org enablement satisfaction that's measured through a company-wide survey every six months.

This helped us have a baseline to make sure that we're allowing people to quickly move through product ops processes, and also to allow us to gather a lot of qualitative feedback.

Secondly, for each of the experiments we run, or new initiatives, we make sure we set out success criteria for those and then ladder up to broader focus areas that we had described before, like the voice of the customer, strategy and planning, building and launching. For each of those different areas, we have our perspective.