Metrics, funnels, KPIs - a comparative introduction
I know, I know - startup metrics have been preached about for years by luminaries like Mark Suster, Dave McClure, Eric Ries and many, many others. The field is full of great companies and tools like KISSmetrics, ChartBeat, Google Analytics, to name but three (and do a great disservice to many others). Companies like Facebook and Zynga collect and analyze oodles (that's a technical term) of data on their traffic, customers and products, and have built multi-billion dollar business on metrics. Surely everything is done already, and everyone not only knows that metrics matter, but also how to select the right metrics and implement a robust metrics process? There's nothing to see here, move along... or is there?
Metrics depend on your business as much as your business depends on them. No, more, in fact. It is possible (though hard) to build a decent, if not awesome business purely on intuition, but it is not possible to define metrics without understanding the business. Applying the wrong metrics is a disaster waiting to happen. In fact, in some ways this makes building a robust metrics platform more difficult than building the product it's supposed to measure. Metrics can't exist ahead of the product, but are needed from the beginning. Sure, with experience you will learn to pick plausible candidates for KPIs, and may even have tools ready for applying them to new products, but details change, and sometimes, with those details, the quality of the metrics changes dramatically. This is obviously true between industries like retail vs entertainment, but it's also true between companies working in the same industry.
This is a big part of why metrics aren't a solution to lack of direction. They can be a part of a solution in that well-chosen metrics will make progress or lack of it obvious, and may even provide clear, actionable targets for developing the business. Someone still needs to have an idea of what to do, and that insight feeds back into all parts: product, operations, measurement. I've never liked the phrase "metrics driven business" for this reason. Metrics don't do any driving. They're the instrumentation to tell you whether you're still on a road or what your speed is. You still have to decide whether that's the right road to be on, and whether you should be moving faster, or perhaps at times slower.
What to do, then? Well, understanding the differences helps. Lets start with a commonly applied metrics model, the sales funnel.
In a business-to-business, face to face sales driven business, a traditional funnel may begin with identifying potential customer opportunity, then measuring the number of contacts, leads, proposals, negotiations, orders, deliveries and invoices. A well managed business will focus on qualified customers and look for repeat transactions, as the cost per opportunity will likely be lower, and the revenue per order may be higher, leading to greater profitability. They will also look at the success rate between the steps of the funnel, trying to improve the probability of developing an opportunity into a first order.
This model is often adapted to retail: advertising, foot traffic, product presentation, purchase decision. For some businesses that's it - others will need to manage the delivery of the product, and may see further opportunities in service, cross-sales, or otherwise. Online retail businesses measure every step in much greater detail, simply because it is easier to do so. Large retail chains emulate that measurement with very sophisticated foot traffic measurement systems. But even in its simplest form, while the shape is similar, the steps of the funnel are very different.
Online businesses have developed a variety of business models, among which two large categories are very common: advertising and freemium.
The advertising-funded two sided market model is two different funnels: a visibility - traffic - engaged traffic - repeat traffic page view model, and a more traditional sales funnel for the advertisers, though even that one has been, through automation, turned to look more like an online retail model than what advertising industry is used to. This model is further enhanced by traffic segmentation and intent analysis, allowing targeted advertising and a real-time direct marketing product, the sales funnel of which is even less familiar to the sales funnel I described at the beginning.
Freemium isn't even one business model: a B2B service with a tiered product offering and a free time- or feature-limited trial product may ultimately use the traditional sales model, only so that the opportunity to prospect part of the model is fully automated. Often it's entirely automated to the point a customer never needs to (or perhaps even wants to, assuming a simple enough product) talk to a sales rep. Still, the basic structure holds: some of the prospective leads turn into customers, and carry the business forward. Free service, be it for trial only or for the starter-level segment, is a marketing cost and a leads-qualifier, enabling a smaller sales force.
On the other hand, the free-plus-microtransactions model, one which we pioneered with Habbo Hotel, and has since been used to great success by many, including Zynga, can certainly be described as a funnel, but to measure it with one requires significant violence to many details. The most important of these is that because individual transactions are typically of very low value, building a profitable business on top of a model which aims for, and measures one sale per customer is practically impossible. This class of business doesn't just benefit from repeat customers, it requires them. Hence, a free-plus (or, as it is called in the games industry, free-to-play) business model must replace counting a "new customer" metric or measuring individual transaction value with the measurement of customer lifetime value. Not just measuring it on average, but trying to predict it individually - both to try to develop 0- or low-value users (oh, how I hate the word) to higher value by giving them better value or experience, and by identifying the high value customers to serve and pamper them to the best of the company's ability (within reason and profitability, of course). And once you switch the way you value revenue, you really need to switch the way you measure things pre-revenue.
Funnels change. There are business models where funnels really can't provide the most instructive KPIs, even though they still may be conceptually helpful in describing the business. As this post is getting long, more on the details of KPIs of free-to-play in the next episode.