Today we’re going to be talking about product metrics. This is a topic that you’re really excited about, We’ll be talking about north star metrics and how do you create north star metrics, what does that mean for your business.
You’re obviously very interested in data and how we can make better decisions using data. How do you think about product metrics?
Yeah, for sure. To me, product metrics are all about decision making. They should be able to enable your organization to make better decisions faster. I kind of think about that along two axes. First is strategically. It’s, “Are you tracking the right metrics?” I think about that as I want my metrics to tell me are customers being able to, I guess, complete the use cases that they’re coming to our product to do.
Operationally, that’s all about, “Are the metrics easy for everyone to interpret, to read? Is everybody on the same page? Do they know where to find them?”, but more of like actually once you have the metrics, what do you do with them? Yeah, because you have to do something with that data, otherwise it’s not operationalized. I hope people do stuff with it, at least. Yeah. What’s the difference between a product metric and a KPI?
I actually think of these as fairly similar. At least, I use the two terms pretty interchangeably. Do you think that PMs are the best person to be the owner of product metrics? I actually do think PMs should be the owner, but it needs to be a collaborative process. What I mean by that is I’m definitely a firm believer that if everybody owns something, no one owns something. Kind of a tragedy of the commons type thing. Like I said, it needs to be collaborative. There are some things that may be like an engineer or a designer might understand a little bit better than a product manager. Product managers should probably work with their customer success teams and their support teams to get a better understanding of what their actual customers are saying when they start to develop metrics that they’re going to actually make decisions off.
Product metrics are important because they help create that alignment between all these different organizations at your company. Without product metrics, it’s hard for everybody to speak on the same page. It’s hard for people to prioritize things if everybody is kind of focused on different metrics or different ways to think about things.
So being able to prioritize things quickly and effectively is super important. What makes a good product metric versus a bad one? Is there a bad one? I think there are probably certainly bad ones, but it also depends on what you’re trying to track. Sure. Churn, to me, is actually an example of potentially a great product metric, like what better way for someone to tell you that they’re not getting use out of their product for them to actually stop using you and churn? But it can also be a bad product metric because if I’m looking at churn, it’s kind of too late.
I would like to know that customers or users aren’t completing their use cases well before they actually make the decision to quit. Yeah, so you need some of those leading indicators. Let’s talk a little bit about growth. How do you think product metrics change throughout growth?. Product metrics definitely can and should change as the company grows, and I definitely want to reiterate that it all goes back to the customer and how your product is evolving to meet your customer’s needs, every day to analyze different customer trends, watch sessions of people encountering bugs, etc.
That was something that we said daily active users is very important for us at that time. As our product has evolved a little bit, and as we
start tackling new use cases, such as conversion rate optimization, product manager use cases, we decided that daily active user isn’t a necessity for these users, so may be weekly active is the right way to start thinking about this. There’s definitely never going to be like a light that comes on that says, “It’s time to change your product metrics.” Nothing is black and white, but I think the perpetual, constantly reevaluating who my customers are, how are they most getting value out of my product, is the best way to really decide what metrics should I be tracking.
If you have any thoughts on product metrics, either ones that you think are really great or other ideas about how you measure engagement with your app, please feel free to post those in the comments below.
How do you keep your metrics fresh? it’s kind of like what are the customer interactions. You need to be very tune with like what your customers are doing and what are the most important features and what are the core problems they’re trying to solve. That’s certainly key. We’ve actually developed a north star metric, When you have a north star metric and you actually take the time to socialize it and weave it throughout the fabric of your product team, it really can be enlightening in terms of like, “Oh. A metric that we have is going down or up, Data is certainly the great equalizer.
That way, when I see my north star metric go up and down, I can kind of immediately pinpoint, “Oh, monthly active users is down. Kind of thinking about this in the positive light of what happens when your north star metric goes up, this is something that we look at really closely when it comes to things like feature launches or big, drastic changes in a product. we launched a feature called Custom Events. We noticed that our customers that were using the feature were actually performing better on the north star metric than our customers who weren’t. That in and of itself was like, “Great. We released something that we wanted to change our north star metric, and it did.” What was really enlightening for us was actually looking at the KPI tree and figuring out why the weekly active explorer numbers were going up. That was something that was like, I guess, a great product metric win for us, being able to predict what would happen and then actually see it happen in our metrics.
If someone wanting to get started maybe creating a north star metric or becoming more data-driven, what advice do you have for them?
It was like making sure that we had the data needed to analyze the things I wanted to analyze or start to like string all these things together, eventually bubbling up into a north star metric. A lot of times at early stage companies, you might not necessarily have an entire BI or an analytics team that’s sole focus is to work on an analytics stack that people can query and draw inferences from. I would say it’s understanding what data you have and then how to get the most value out of it because even though getting the data’s important, I think you’ll find pretty quickly that there’s a plethora of things you can analyze. It’s really about asking the right questions and figuring out what you want to track. I think that all goes back to the customer, trying to understand who are the people that you’re trying to serve and what do I need to know to best serve them.