Show Notes
In this episode of Startups For The Rest Of Us, Rob and Mike talk about the one metric that matters. The idea of focusing on one metric at a time, the metric that is helping move your business forward.
Items mentioned in this episode:
Transcript
Mike [0:00:00]: In this episode of “Startups For the Rest of Us,”
Rob and I are going to be talking about the one metric that matters. This is “Startups For the Rest of Us” Episode 260.
[Theme Music plays]
Mike [0:00:16]: Welcome to “Startups For the Rest of Us,” the podcast that helps developers, designers and entrepreneurs be awesome at launching software products. Whether you’ve built your first product or you’re just thinking about it. I’m Mike.
Rob [0:00:23]: And I’m Rob.
Mike [0:00:24]: And we’re here to share our experiences to help you avoid the same mistakes we’ve made. What’s going on this week Rob?
Rob [0:00:28]: MicroConf 2016 in Las Vegas, the dates are set. We signed a contract, although we haven’t received the counter-signed as of today, but as far as we know, everything’s in place. Las Vegas, April 3rd through 5th. So if you’re interested in hanging out with about 200 of your closest founder friends, a lot of bootstrapped founders heading over there out in early April. Go to MicroConf.com, enter your email address. The tickets every year sell out way before they make it to the open market. So if you are interested in coming, you want to get on the list.
Mike [0:01:01]: You know, it’s always a little disconcerting talking to people about the dates before we have that counter-signed contract.
Rob [0:01:06]: I know, I know. I’m such a more conservative in terms of stuff, and I like to have everything signed and known, but boy, we’ve been talking to the hotel for months and the odds are pretty good it’s going to go through.
Mike [0:01:17]: Yeah, I think I even remembered that there’s a clause, kind of early on that they kind of throw in there that just says that if they kind of offer us right of first refusal. So if somebody else comes in and says, Oh, we want that date, they have to come to us first and offer it to us, or at least try and get something signed. But because we don’t have a counter-signed contract, I guess it’s, I don’t know, I guess it seems less official.
Rob [0:01:39]: Sure. Well it is, it’s not legally binding at this point. So feasibly there could be a screw up at some point, where they get our signed contract and go to hand it to whatever the VP in charge of signing contracts, and someone else hands him one for the same dates, and they’ve just totally screwed up. It’s within the realm of possibility. But knock on wood, this is what our eighth or ninth conference? And we haven’t ran into any major issues like that. How about you, what’s going on?
Mike [0:02:02]: Well, I’ve been doing a little bit more customer validation on the new idea I’ve been working on, and that’s been going well. I’ve had a couple more conversations this past week. And right now, something else I’m working on is I’m testing whether or not it makes sense for us to move Founder Café off of the Communifire platform and onto Discourse. So I don’t know how that’s eventually going play out, but so far it seems to be going reasonably well. I don’t know all the details of it. It’s kind of the unfortunate part. I know about half of it, because I don’t know the Discourse side of it, or Rails or Postgres. But I’ve got all the Communifire stuff that I’ve been working on. I’ve been working with a contractor to help with the other side of things. And we’ll see how things work out. I’m not real sure it.
Rob [0:02:45]: All right, cool. You mentioned that you’ve been doing a small amount of validation for your new idea. How come so little? I would imagine that you’d be fired up, cranking away on it.
Mike [0:02:55]: So what I did, when I was going through my validation steps, was every conversation I was having with people, I would basically just take a ton of notes during the conversation. And I’m starting to get, probably more towards the end of my list of people that I’m talking to, and I’m starting to hear the same types of things over and over again. So I’m going back through and I’m probably having less conversations now than I have in the past couple of weeks, but it’s more because I’m sifting through all that data I have to try and find overlap between different people and figure out exactly where to go. And I’m at the point right now, where I’m probably going to start – I am at the point where I’m starting to kind of draw up designs for what it’s going to look like and then take those designs back to them as kind of a double check.
Rob [0:03:37]: Is that your next step?
Mike [0:03:38]: Yeah, that’s what I’m working on right now is, as I said, just sifting through the things that they’ve told me, figuring out where those overlaps are and seeing what the lowest common denominator is that I can put in front of them that they’ll say, that they’ll still continue to say, “yes, I’ll pay for that.” And then kind of get the commitments from everybody to move forward.
Rob [0:03:55]: What’s your ETA on doing that?
Mike [0:03:56]: I’d probably say, I started working on the designs this past week, and it’s taken longer than I expected it would to go through some of those designs. So I want to say one week, but I’m guessing it’s probably going to be closer to two or three.
Rob [0:04:10]: Those things are always hard to estimate, kind of like building software. We’re working on some pretty cool stuff inside the Drip walls, and our end growth is continuing like as it has been for the last several months, things are going well. We’re actually hiring another developer, more of a front-end emphasis. And we just have so many features that need to get built, and now there’s even marketing stuff that I want to do where you just need some code and you need some design work and you need some Javascript and some CSS, so that it’s just enough that I can’t sit down and bang it out and I can’t get everything else done that I’m trying to do, so we’re probably going to hire someone and have him or her help do the majority of their time in hardcore development, but then also helping out with kind of marketing tasks, because that need front-end work and development.
Mike [0:04:58]: Cool. So this week, what we’re going to be talking about is the one metric that matters. And this is inspired by a couple of articles that I’ve come across online, one of which is from the Kissmetrics blog, and the other one is from leanstack.com, and then a third is from the leananalyticsbook.com. And all three of these articles refer back to the one true metric that matters. And essentially the idea here is that you should only focus at one metric at a time. And that metric is the one that is supposed to be moving the needle in your business. And we’ll link up all three of these articles in the show notes. But again, going back to this point, the idea is that you can track other metrics, but there’s one that’s going to be the most critical to you, which is going to depend a lot on the goals that you’re trying to reach at that time. So for example, the one key metric that I’m tracking right now is the number of prospect conversations per week. And because I’m kind of moving into the design phase, that’s probably going to change. But that’s the one metric that I was tracking over the past three weeks.
Rob [0:05:55]: And for me, I haven’t given it any thought in advance of recording this episode, and I guess we’ll see as we go through this if mine fits this criteria because I know we have some kind of does and don’ts of how to do it, but I would say it’s MRR for me. And so you might look at that as MRR growth, how much did it grow this month and is it going to grow more next month than last month. But really the one I look at is just MRR since it is a recurring metric itself, that’s what I have tended to focus on. And I think if I were to commit, of course, there’s everything then falls below that, right, of I could look at the trial count for the past 30 days. I could tell you if we’re going to grow and by how much at this point at the trial. The paid conversion and what we’re going to turn out and the upgrades and all that stuff. But the real focus that all feeds into, MRR.
Mike [0:06:39]: So the first thing we’re going to talk about is why? Why is it that we’re only looking at one particular metric? And the first thing to keep in mind is that these metrics encourage focus. If you only have one metric that really matters, then you can essentially ignore everything else that’s going on, whether – and in your case, for example, you’re tracking MRR and that’s the one metric that matters the most, then you can essentially ignore all these other things like the conversion rate and the number of visitors to your website. A lot of those things can just go right out the window because they don’t necessarily matter. And sometimes these metrics will play into one another and influence each other, so that’s something else to keep in mind when you’re looking at these metrics. And it’s especially important when you’re looking at something like MRR because all those things do essentially influence MRR, but not directly.
Rob [0:07:26]: Yeah, I would agree with that. I mean, I think being able to focus on one thing and not be distracted by trying to chase after a lot of things all at once is going to be a good thing. One point, early on, when Noah Kagan was building AppSumo, I had talked to him and we were talking about kind of what you’re focused on, it was like focusing on one thing. And at the time, I think I was doing, it was early in HitTail, and I was talking about, yeah, it’s just MRR I was focused on. And he said, I’m focused on growing my list. He said, it’s the number of subscribers total and the number of subscribers we’re adding each week or each day. And I remember thinking, oh, how interesting that he wasn’t focused on – there was not, he wasn’t focused on revenue, wasn’t focused on profit, wasn’t focused on deals per month, there’s a bunch of other things that you could look at, but at that point he knew that to get where he wanted to go, he had to build that list. And that’s kind of why I look at MRR as SaaS founder, is that to get where I want to go, I need to get that to increase. Now there’s a bunch of ways to do it, but it does give you focus. The other thing is it tells me what not to look at. As an example, I don’t, at this point, when we’re in heavy growth mode, I don’t look at profitability. Now, we are a profitable company, but I’m not measuring my success based on how much cash we can make in a month on a net basis. I am looking at that growth number as long as we are not doing stupid things like paying more than our customers’ lifetime value in order to acquire them or something like that. If you keep that in mind, I think that focusing on that one thing and then looking at all things that lead to it is a good way to go.
Mike [0:08:49]: And that’s the second on the list of why you should be looking at just one metric, because it helps narrow the field of things that you need to pay attention to. And even if you can’t pick just one, if it really boils down to like two or three, then at least it’s eliminated a lot of the other things that could potentially be distracting. If you’re looking at too many of these different metrics, essentially what happens is it just becomes noise, because it’s hard to get everybody to look at the same thing at that particular moment. If you have everyone looking at the same thing, then what it does is it helps create accountability for everyone. Everyone’s all focused on the same goal, and it becomes very easy to relate the different things that people are working on to the projects or the tasks that other people are working on.
Rob [0:09:30]: This also creates accountability for your team because folks can kind of focus on a single metric and then all be working towards a higher level metric. When I say that, to come back to the MRR example, that as a founder of a SaaS company, that’s what I’m looking to grow. But perhaps the person sitting next to me is working in marketing, her goal may not be to grow MRR directly. Her goal may be to drive more trials. I know that is going to feed towards the company goal of increasing MRR. But then she doesn’t have to look at her job and say, boy, how did I myself increase MRR this month? She can just say, I know I drove this many trials and directly measured that based on her efforts, so I think that having a single company goal as well as individual goals for each person or kind of individual metrics based on what they’re working on that they can directly influence, that all lead up to that company-wide goal, is I think a good way to do it. Or it’s at least something that we found success with at Drip.
Mike [0:10:24]: It also encourages people to kind of experiment within the guidelines of that because you can always look at something that is not necessarily related to that KPI and say, well, I’m going to do this A-B test over here. And that could be very well be optimize a specific part of your sales funnel, but it’s not necessarily related to your goal. And it’s not to say that doesn’t help the business in some way, shape and form, but if it’s not what you’re focused on, then essentially what you’re doing is you’re spending time improving something that isn’t directly related to the company’s current goals. So the next part of this process is figuring out which metric matters. And essentially, there’s four tenets of how to pick the metric that does matter. There are four pieces of this. There are things that you know, which are essentially facts. There are things that you don’t know, which are questions that you have, which you know the questions to ask. Then there’s things that you don’t know that you know. And those are essentially things that are driven by your intuition. You have a gut feel about something. And then the last one is probably the most mysterious of all these is those are the things that you don’t know, that you don’t know. And a lot of times if you start exploring these areas, this is where you’re going to find those unfair advantages. You’re going to uncover things that other people in the market haven’t really figured out yet, and those could substantially drive your business forward.
Rob [0:11:39]: I’m not sure I understand these four in terms of how they relate to picking that metric.
Mike [0:11:45]: So the idea with these four tenets is that essentially what you’re trying to do, is you’re trying to distill all of the things that you do know and that you don’t know into a mechanism for identifying the analytics and the metrics that you should be looking at. So for example, if you are able to directly measure what is going on in your website, the number of visitors that are coming, then you can interpret those as facts. So if you have Google Analytics installed you can, with giving some credit to Google for actually being accurate on all of this, you can look at that and say well, yes, I know that I’m getting ‘x’ number of visitors per month or per week to my website. So those are the facts that you know. And then there are things that you don’t know that you could answer by reporting, by saying, okay, well, we know that we’re getting this number of visitors to our website, how many people are coming over and viewing our pricing page, or visiting our pricing page and then going in and signing up for a trial? And for something like that, you might use Kissmetrics or Mixpanel or something along those lines. But those are questions that you have that you could ask that you just, you know the question, you just don’t know the answer because you haven’t gone and looked at the data yet. And then there’s things that you don’t know, that you know, which are things that you just have a gut feel about. So for example, you might say, well, I think that we’re getting a lot of our trials coming in through word-of-mouth. If your products are growing and your customers are telling other customers about it, then you might have an intuition about that. But you don’t necessarily know that, and it can be very difficult to figure out exactly how that’s happening without doing a lot of analysis of those people that are coming in and having direct conversations with those people one-on-one, for every single person who signs up for your service. And then the fourth category are the things that you don’t know, that you don’t know. And those are things that, honestly it’s going to change a lot based on your type of business, but you might find out that maybe you believe there is a viral component or that people were sharing a lot of information about, oh, you should sign up for this service and your customers were talking to other customers about it. But you may just find out that, oh, it was posted on some forum someplace, and you didn’t even know that that was a great way for people to find out about your service.
Rob [0:13:53]: Okay, cool, that helps clear it up. Finding the metric that matters also depends on what kind of business you’re in. If you’re running a SaaS app, maybe it’s MRR, maybe it’s MRR growth. Maybe if you’re at an earlier stage, it’s just trying to get some customers in the door, customer count. If you’re like in a collaborative or a community software company, maybe something like StackExchange or Reddit, then maybe it’s the number of votes, the number of comments, the number of new content pieces that are created for you by the community. If you’re a media company, maybe it’s the number of page views that you’re getting. If you sell games, maybe it’s the number of in-app purchases. Or if you sell apps and maybe don’t have direct access to your customers, you’re obviously going to have less data than if you’re dealing directly with folks, so maybe you sell a mobile app through the iOS app store, then you might have to look at a higher level thing, like the number of downloads of your free version and then the number of purchases or number of dollars coming from your paid version. And finally, if you have just a transactional business with like one-time sales, so maybe you have desktop ftp client, or you have invoicing software that someone can download and install and it’s not a subscription, obviously your metric is going to be different than a SaaS app, because you don’t have monthly recurring revenue. At that point, maybe it just is total revenue per month of total downloads of your free version, if those tend to highly correlate with getting more downloads and more sales of your paid version.
Mike [0:15:20]: The other thing that factors heavily into this is what stage of your business are you at. So are you really in the early, early stages where you’re just talking to people and you’re just trying to get attention for a landing page that you’ve put up. Are you actively having conversations with people and doing a needs discovery? Are you at the validation stage, where you are creating an MVP for the product? Or have you even gotten past that point and you’ve launched the product and now you’re looking at doing feature optimizations and implementing customer requests, or even further than that where you’ve got a stable app and you’ve hit product market and you’re really just trying to optimize the entire business, based on which of those stages you’re at, your metric that you’re looking at is going to change kind of dramatically from one stage to the next.
Rob [0:16:03]: Yeah, absolutely. I mean, if you’re pre-launch, then I’d say a big metric is how many email addresses can you get on that pre-launch list. If you have a few customers, beta customers in there early and you’re doing customer development, maybe it’s the engagement, maybe it’s like the amount of times they log in or the amount of features that they’re using. Or even like the number of features that they suggest that kind of fit within your vision. I mean, it could be as simple as that, when you’re trying to get to that next step, it actually comes easier and more clear once you’ve broken past, I’ll say product market fit, but once you know that you’ve built something that people want, it becomes easier because then you tend to just have a pretty straightforward metric that you’re trying to grow. It’s going to be probably revenue or trial downloads or there’s something there, but before that at each stage it’s going to be changing pretty frequently as you move between the stages.
Mike [0:16:53]: Right, I remember kind of a specific example from when Facebook was kind of building up in the early stages. One of their metrics was that they wanted somebody to sign up and add, I think the number was like 10 or 20 friends, within seven days. And those people were going to have a dramatically experience and be more “successful” with their product than the people who did not. So that was one of their key metrics that they used very early on. So it was very feature driven at that point. They said if somebody uses this particular feature which is just adding friends, then they will be more successful with the product, and we want to be able to identify those people and figure out how to get more of those people. So the next thing to think about is who is this metric for, you know, who is the intended audience for this metric? If it’s MRR, then it might be to management, but there are different metrics that you can track that could go to internal groups. It might go to the marketing team or the developer team. You might have metrics that you’re developing for investors or for the press, or as part of a marketing campaign. There’s lots of different reasons why you might have these metrics, but you want to be able to make sure that you are identifying these metrics for a very specific reason. And you want to know who those metrics are going to, because that’s going to make a difference. And the specifics of which metrics you’re tracking.
Mike [0:18:10]: So let’s start talking about what makes a good metric. And the first part of this is make sure that your metric is a rate or ratio, because those are going to be better than an absolute or a cumulative value of any kind. So, I’ll give you an example, from earlier in this particular podcast I talked about the fact that I was measuring the number of conversations per week. Now, if I were just measuring the total number of conversations I had, then that wouldn’t necessarily be a good metric. But the fact that I’m measuring them on a per week basis, I’m able to relate one week to the next and figure out whether or not I’m maintaining progress or declining or exceeding the expected rate.
Rob [0:18:51]: And I think it’s interesting to think about with the SaaS app, we never even talk about what’s the total aggregate revenue that the app has ever generated, right? That would be a very bad metric, that’d be an absolute number and it would grow every month, right, because as long as you have revenue coming in, you would just add it to it. And that number could be hundreds of thousands of dollars or millions of dollars, but that doesn’t give you a really good picture of what the business is doing because there’s no rate or ratio to it. So I think, if you think about MRR as it’s basically the recurring revenue that you have at the end of that one month, so that gives you kind of a rate there, and then I think growth, like MRR growth is a ratio as well, right, because it is the amount of new revenue, new MRR you’ve added compared to your total. And so, I know that a lot of start-ups look at their month over month percentage growth, and that’s a big thing that [YEC?] looks at and a lot of these accelerators look at. I personally don’t track, I’ve gone back and calculated it for particular purposes when the audience, whether I’m giving a talk or whether I know someone is actually looking for that number, it’s easy enough to calculate going back, that is not a key driver that I look at when managing the business, but that’s not a bad thing. I think if you are a startup like Paul Graham says, growth is everything, and so for them, even if you’re doing revenue growth, they probably lose sight of the absolute revenue because that’s less important than the growth and the rate of growth that they’re experiencing as they’re working on it. Because growth has a lot to do with whether or not you have traction. And traction has a lot to do with whether or not you’re going to be able to raise that next funding round. So if you’re in the V.C. space, the venture funded space, about every 18 months you tend to have to raise a round. And even if your MRR is going up, if your growth numbers don’t hit the ranges that they need, it’ll be pretty hard for you to raise a round. So you can see how for those guys, given their audience, they’re tracking more growth, whereas someone who’s maybe boot-strapping a business, and probably has other goals, might look at something a little more solid like monthly recurring revenue or some people might even look at monthly net profit.
Mike [0:20:56]: But I think the key point that you’re making there is that those are monthly numbers, that it’s monthly recurring revenue or monthly profit. And it’s important to have that monthly piece of it, because if you don’t have a time period of any kind, then really it’s just a number, it’s a vanity metric at that point, which is essentially meaningless to the business. You can’t compare it to other time periods because it’s just a number. If you are able to compare it to other time periods, that’s when it becomes meaningful, that’s the important piece, and that’s what makes it a good metric.
Rob [0:21:25]: Yeah, it also makes predictions a lot more easy to come up with and probably more accurate. If you’ve been tracking this number over time for these small time periods, so let’s say weekly growth or monthly revenue, you get a sense of where this thing’s going. You can notice pretty early on if you’re looking at the rate of change, you’ll notice on a graph or even intuitively in the numbers, if you’ve been following this, you can kind of feel the pulse of it. And you’ll notice as things are changing, you’ll be able to predict out a month or two. I don’t tend to do predictions for predictions sake, but there are a few conversations I’ve been in where people have asked me, now where do you think you’ll be in three months or six months or whatever, and I have a decent idea. And so far the predictions that I made during the summer were hitting those, barring some unforeseen circumstances, you can get pretty good, if you’re doing this ratio per time period, you can project out and be reasonable as long as a major roadblock that’s unexpected doesn’t crop up.
Mike [0:22:18]: Another thing that makes a good metric is that it’s easy to understand. So for example, monthly recurring revenue, is very cut and dry. You can easily understand that. Same thing when it goes to like profit or conversion rates. But I think that once you start getting into some of the super advanced metrics that are much more difficult to understand, so if you’re aggregating a bunch of data, and then trying to use that to compare against other aggregated data over different time periods, as soon as it becomes a lot more difficult to understand, it becomes much more difficult to also figure out what it is that you need to do in order to start making changes to that number or how to influence it. So you want to try and choose a metric that’s simple and easy for everybody in the organization to understand so that they know what their capabilities are around influencing that number.
Rob [0:23:06]: And another piece that makes a good metric is if the metric helps you make predictions more accurate. I already touched on this a little bit, but I kind of went about it the other way, saying that if you have a good metric, it makes it more accurate. But you’ll want to choose a metric that actually helps you make better predictions. I think that’s a key piece. If you find that yours is not, then it’s probably not a very good metric.
Mike [0:23:30]: And the last thing that makes a good metric is that, the metric has to change your behavior. And one of the things that I’ve seen a lot over the years is that when people are doing A-B testing, for example, they’ll do A-B testing just to do A-B testing. And it doesn’t change what they do. They’ll pick it up, they’ll do it for a little while, and then they’ll stop. And whether they see results or not, the fact is they don’t carry that forward. So it doesn’t change their behavior. So the fact of the matter is, like why are you even bothering. If those numbers that you’re getting out of that aren’t going to change or influence what you do, then why are you even tracking that. It’s essentially immaterial at that point.
Rob [0:24:05]: If you’re interested in learning more about finding your one key metric, we have three links for that we’ll put in the show notes that Mike used to help put together this episode. And if you haven’t read it, you’ll probably want to check out “Lean Analytics.” It’s a book from O’Reilly Press. It’s written by Ben Yoskovitz and Alistair Croll. And it is worth looking at, if you are looking for the single metric that matters. If you have a question for us, call our voicemail number at 888-801-9690 or email us at questions@startupsfortherestofus.com. Our them music is an excerpt from “We’re Outta Control” by MoOt, used under Creative Commons. Subscribe to us in iTunes by searching for “startups,” and visit startupsfortherestofus.com for a full transcript of each episode. Thanks for listening and we’ll see you next time.
I miss Marla
Great to hear you guys talk about Lean Metrics. Just to add to a list of resources. Ash Maurya from Lean Stack and Running Lean did a talk at Leanconf 2014 called “One Metric To Rule Them All” – http://2014.leanconf.co.uk/videos/one-metric-to-rule-them-all-ash-maurya