This panel session was held at the Product-Led Summit, London 2019.

Speakers

Drake Andrews, International Growth Manager, Shopify

Eric Peters, Product Owner, HubSpot Academy, HubSpot

Paul Holmes-Higgin, Chief Product Officer, Flowable

Jamie Norman, Head of Product, RedEye International

Identifying the ‘Aha!’ moment in your product

Q: Do you have a clear 'Aha' moment for your product or company?

Jamie: "There's an 'Aha!' moment for our customers which is usually quite analytics-led. When they see the results of their first campaigns and they take that on board, they can leverage that to iterate and build on their marketing campaigns.

But, we can look at this from a different perspective as well. We have our own 'Aha!' moments looking in on our customers. One of those is how we build our roadmap. We build a roadmap based on broad time horizons. We share this with our customers and we show what we're working on right now. We've got a column for what we're doing next, and we've got a column where we share what we're planning to do in the future. A real eye-opener for us was when we started to share this with our customers and started to use this as a prototype for our strategy. They started coming to us and informing us what should have been higher than it was on the roadmap.

That was compounded by workshops that we ran. I can't overemphasize the insight that watching people use your product gives you. You give them a simple task to work through and you can see where they get stuck. Rather than just reading numbers on a spreadsheet, you can understand the meaning behind them."

Paul: "From my perspective, there are two ‘Aha!’ moments. The first comes from the customer and the second comes from the product management. This is when we suddenly understand how to engage with the user. It's very difficult for enterprise software since we typically give our product to a systems integrator, who will then take it and configure it for their end customer. The link between the product people and the users can be very remote and so capturing that 'Aha!' moment can be extremely difficult."

Eric: "With HubSpot Academy as an acquisition channel, for a long time we focused on the first lesson that the user would complete. We saw if they completed their first lesson within six hours, they had a much higher retention rate going forward for the first few weeks.

When we started running experiments against that activation point, we found there were other factors that could influence it. Maybe the topic that they were learning about, or how engaging that professor was of that course. We took a step back earlier this year, and we did impact outcome mapping. We looked at the overall impact we want to have in the business. Then we looked at what the key success measures of that impact was. Then we broke those key success metrics into the specific user behaviors. From there, we could make a much more direct connection between user behaviors that impacted the overall MRR of HubSpot Academy for HubSpot. We've landed on a couple of different metrics."


Drake: "We've looked at cancellation reasons. A large segment was client’s who didn't make any money. That’s important for a merchant who’s using our product. But, with that, there's a clear way to go about it. We added more resources and value to make sure that these people made their first sale. We made sure our support team was able to communicate and educate every single person that calls us. You might call about setting up your domain, but the support person would do a quick audit of your store and say, 'Hey, did you know that you have new Facebook channels or Instagram channels?'"

Q: How did you arrive at your ‘Aha’ moment?

Drake: "Consider doing a live recording of somebody using your product and study their facial expression. I think that can tell a lot more than a sheet of data."

Paul: "Sadly, a lot of it seems to be from my experience, you've just got to see it. Trying to spot it in data can be difficult. You have to spend time with your customers. That's the only way to truly do it. And if you set out believing that you understand what the 'Aha!' moment is, you've probably got it wrong. You can't invent it."

Q: What's the added value of having an 'Aha!' moment?

Drake: "Making sure everyone's aligned on the team is going to result in more impact for your product. You want to find a certain point in time, through the merchant journey or through your user journey that's going to help them stay on for 5/10 years. It’s about aligning your team and all your growth experiments to lead to that."

Paul: "It's also about trying to get the Developers to understand the 'Aha!' We're very lucky. We're open source. Our Developers have a very close relationship with a lot of the end-users. They will see the user 'Aha!' moment.

But the other real advantage of the 'Aha' moment in your customers is that when that person has this moment, they become an evangelist for your product. Suddenly, they're doing the selling for you."


Eric: "In a freemium situation, you only have minutes for the user to find value. You want to centralize your efforts to make sure that the most amount of people find as much value as possible. Think of it as an ease spectrum versus a motivation spectrum. If it's very easy, they don't need a lot of motivation to get to that moment. For us, we need one central point that our Engineers can focus on and get as many people to that point, so they see the value and stick around to continue evaluating your product."

Q: How do you create new 'Aha!' moments to encourage usage?

Jamie: "We've looked at this in a similar context to A/B testing. You have an ‘Aha!’ moment and then you start to use that as a metric within the business. That becomes a focus for how you're going to build on your acquisition.You might have a regular model or a winning test that you use as your default approach to an experiment. If you think you've got a new moment that might surpass that, because it brings something new to the products, then you can test that against the existing one.

You might have multiple 'Aha!’ moments within your product. Don't feel like you're restricted to just one. If you consider it as a natural customer journey, there might be multiple points along the way where you can uplift new features, or where you see a greater adoption of the product."

Q: How do you collect feedback to find opportunities for improvement?

Drake: "There are two ways to go about that: either asking your users or digging into the data. Both can tell you two different things. The data is going to tell you what's happening. You can see where your funnel is breaking. But talking to people face to face and getting that feedback is going to tell you why that’s happening. You want to ask people who have gone through that funnel what happened at that moment so you can get better feedback."

Paul: "If you're going be data-driven, and especially with the growing use of AI technology, you've got to be aware of profiling. You can’t just look at the data and think that’s telling you everything you need to know. What you end up doing with data analysis is profiling people. If you're not careful, you begin to believe that you're learning something about your entire user base, even when the set of data only rings true for a very small community. This may not be your target community or the market you want to address."

Q: How do you cut features that aren't working for your users?

Drake: "If you're experimenting with different projects or product features, you have to measure the impact and get a good read of your users. There's a lot of notorious examples of this.

Coke, for example, was declining in market share with Pepsi. So they ended up changing the recipe. The public was in outrage. They quickly pivoted within a month. They went back to their old recipe and rebranded to Coca Cola classic. You can have certain tripwires in place that's are really going help you identify when you need to force pivot. Sometimes it's really hard. If you're a Product Manager, you can take things personally."

Paul: "If you have to pivot, then you also have to accept that you've made a mistake. Your assumptions about something was wrong. And you've got to take it personally because you've messed up in some way.

If you have to pivot as a Product Manager, some of the culpability comes to you. And you've got to learn from it.

Sometimes, you may be right not to pivot. 'We're right, and you'll learn to live with it.’

But, you should always have planned out, ‘What if there is a reaction or rejection of this?’ ‘Let's have a Plan B.’"

Eric: "If you think about HubSpot as this ecosystem of applications that all talk to each other, there are 100 mini-apps, and then maybe 50 or so engineering teams. At any given state, we can either be maintaining one of those apps, building it out further or deprecating it.

We haven't had to deprecate many applications. But for the ones we do, it always comes down to the volume of usage versus the pain it's causing in the market. We're creating marketing software, sales software, customer service software. And if consumer behavior has evolved so that an application is no longer needed to do good digital marketing, then there's not enough volume in it for us to warrant having a dedicated engineering team on it.

Calling the dead weight of your application is a very important way to keep it evolving and maintaining value over time."

Q: What's one thing you've done that has had the biggest impact on user revenue or growth?

Drake: "We invest a lot in content. We post multiple blogs a day and try to figure out different ways to go about getting more revenue for our merchants.

Historically, we did a lot of blog content, but in more recent years, we've established our own Academy which is designed to educate and inspire people to try new methods,

We also educate our support. In our example, our merchants talk to us almost every day. We're available 24/7.

Black Friday and Cyber Monday just passed, which was a huge shopping season for our merchants. We have all hands on deck to support these people and to educate them on how to leverage this weekend. We know through our million different merchants that it's the biggest weekend of the year, and some businesses plan the whole year just for this one weekend. So, we leverage those learnings and educate everyone who might be new to commerce."

Eric: "In B2B SaaS, it's about focusing on the customers of our customers. If they're happy, then our customers are going to be happy."

Paul: "For us in enterprise software, the end-user usually doesn't have the option of pressing the cancel account button, whether they like it or not. It's typically not about acquiring users. It is about acquiring the revenue of the business.

From the product perspective, it's ‘How do you demonstrate something aspirational?’ Our more aspirational product features, which aren't going to be used for several years, have shown an innovation or capability of a company."

Jamie: "When I joined Red Eye, Black Friday wasn’t really an event in the UK. In the past 5 years, it’s become the busiest week of the year for us because we have a lot of retail clients. It's no longer just about Black Friday, it's the whole week leading up to it.

We introduced a simple product that had a transformative effect on some of the things our clients were doing. We deal in multi-channel marketing, but Email remains a big part of that. We introduced an algorithm that will allow you to target more engaged people. A lot of our clients could send fewer emails at this time of the year, but of a more personalized nature and still manage to generate more revenue. This kind of insight into the customers of our customers helps to do that."

Q: How do you measure the usage impact of key marketing campaigns?

Eric: "Because we teach inbound marketing, we have to abide by what we're teaching. We try not to run too many ads. The only time we'll run ads is if we have a new course coming out.

We'll look for the query parameters coming in from landing pages and signing up and look at the retention rate. How often do they complete the course? How do they do on the exam? Most of our courses have a certification exam at the end.

We see that people who come in through ads that are less likely to finish the course. They'll score lower on the exam at the end of it. You get this full picture of people's learning outcomes based on the different traffic sources they come from."

Q: How do you leverage partnerships so that more people use your product?

Eric: "We partner with online course providers like Coursera and Udacity. We create content for them that leads people to come and take the certification exam back in our application."

Paul: "Because we're Enterprise, we have partners who are integrator partners, and then we have partners who are technology partners. Typically, the integration partners tend to be more tactical. They’ll find and sell your product for you because it sells their services. The technology partnerships tend to be marketing-driven rather than value-driven.

Also, because we're open source, we don't know about a lot of our partners. A lot of them are just building stuff with our software. We don't track this. As a relatively small organization with a wide global footprint, it's important to have partners that can take your product and customize it to different verticals and regions."

Q: How do you know you need to pivot rather than improve something that is blocking the happy path on the new user journey?

Jamie: "It's usually quite obvious - someone will tell you. We're not a subscription service. So a company would invest in us based on our annual license. We've got a little bit more freedom to review since the feedback loop isn't so immediate that someone would just stop using the products.

RedEye started as an agency model. The tools that we built were used in-house and then we developed those to be public-facing. A lot of our users are still internal users because we still offer a managed service. So, we’ve got very direct feedback on new features before they even go to market. It’s great because we are able to pinpoint problems early on."

Q: For acquisition via content marketing, how do you identify what new content to invest that will lead to the greatest ROI.

Eric: "We look at the monthly search volume. 60 70% of our traffic is coming through Google. If we want to target a search engine results page that we think is valuable based on the product that we offer, we will target that page and do everything we can to rank at the top of it. There's an SEO course in HubSpot Academy you can take. That’s how HubSpot ranks number one when it wants to."