Outlier is automated data insights for your entire business.
Book a Demo
Outlier is automated data insights for your entire business.
Book a Demo
Outlier is automated data insights for your entire business.
Book a Demo
Outlier is automated data insights for your entire business.
Book a Demo

Advanced KPIs

Advanced KPIs

This is part 1 of a 5 part series on Advanced KPIs.

Over the past few months we’ve covered many different kinds of metrics, ranging from financial metrics and user engagement metrics. So far I’ve focused on metrics that work across many different kinds of businesses to make it as useful as possible to as many people as possible.

The reality is that you likely have some more complex metrics that are specific to your business. It might have to do with the way your business is structured, how you generate your revenue or how investors evaluate your performance.

While most businesses have these kinds of metrics, they can be dangerous because they are specialized and complex. Without a larger group of companies to compare yourself to, it might not be obvious when these specialized metrics are biased or incorrect, or if they aren’t the right metrics to follow at all.

This week we’ll review some commonly used advanced KPIs for a range of different businesses, and discuss their pros and cons.

This is not meant to be a comprehensive survey of all KPIs for every type of business, but rather a series of examples on how different kinds of business measure themselves. My hope is that, by the end of the week, you are in a good position to think critically about the advanced metrics you use for your business!

“Any darn fool can make something complex; it takes a genius to make something simple.”

Advanced KPIs: Software-as-a-Service – Magic Number

This is part 2 of a 5 part series on Advanced KPIs.

As we discussed previously, Sales Efficiency can help you make decisions about your sales & marketing spend by measuring how much revenue you generate from every $1 you spend. (Hint: If you generate more than $1 you should spend more.)

However, it can be hard to calculate Sales Efficiency if your company is a Software-as-a-Service (subscription). For subscription businesses it is hard because you will not know how much revenue you generated from January sales until 12 months later! This is why SaaS businesses use a metric called the Magic Number.

You calculate your Magic Number by taking the amount of incremental revenue you generate in a given quarter, multiply it by 4 and divide it by the sales and marketing spend for the previous quarter. For example, if you spent $10k on sales and marketing in Q1 and generated $5k of incremental revenue in Q2, then you would calculate your Magic Number as:

That sounds like hocus pocus.

Ha, that’s a pretty good pun. Seriously, though, you are right that it seems dangerous to extrapolate revenue by simply multiplying by 4. Let us break down the strengths and weaknesses of the Magic Number.

The Good

  • You can calculate the Magic Number and make decisions quickly, before you know the Lifetime Value of your customers (LTV). This makes it useful in the early stages of a product or service which might not have years of data.
  • Subscription businesses are typically very stable, so simple multiplication can be a fairly accurate projection of future value. In many SaaS businesses, the Quick Ratio projection can be over 90% accurate.

The Bad

  • If your customer churn is very high, you may not actually have customers pay for 12 months so you would vastly overestimate incremental revenue. In that case, you would use a more appropriate (lower) multiplier than 4.
  • If your sales and marketing strategy is shifting, you may not be able to attribute all the incremental revenue in a given quarter to the spend from the previous quarter. You should fall back to simpler measures of Sales Efficiency in this case.
  • If you don’t have a very clear definition of incremental revenue, you can misattribute revenue you would have earned anyway to your sales and marketing, which will overestimate their effectiveness.

Even with these dangers lurking, the Magic Number is a useful compass when making decisions about increasing your sales and marketing spend at subscription businesses. It is also a useful yardstick to use to compare how well your sales and marketing is performing compared to other companies in your industry.

Read more about Magic Numbers across different companies.

“Any sufficiently advanced technology is indistinguishable from magic.” 

Advanced KPIs: Markets – Gross Market Value

This is part 3 of a 5 part series on Advanced KPIs.

Marketplaces are everywhere these days. Some are easy to spot (Ebay, Airbnb) and others look more like products than marketplaces (Uber, Amazon). A marketplace is simply a place that connects buyers and sellers. Instead of making money from selling things directly, a marketplace makes a small transaction fee on every sale between a buyer and a seller.

It’s a great type of business when you reach high volumes of sales, but a bad business if your volume is small. The fewer the transactions, the less money you can make. For this reason, many marketplaces use their Gross Market Value (GMV) as a key metric. GMV is simply the total value of all goods or services sold in the marketplace in a given period of time.

Wait, it’s that simple?

Yes, but just because it’s simple to calculate does not mean it’s simple to understand or simple to use. As a smart, data driven person you immediately realize the GMV has some key flaws. Let’s jump into a quick analysis.

The Good

  • GMV is an overall total, combining both high-volume / low price transactions and low-volume / high price transactions. Your GMV goes up if a lot of low priced items are sold or if a few high priced items are sold, both of which are useful measures of growth.
  • It’s easy to compare two marketplaces based on GMV even if they sell vastly different products at different prices at different volumes.

The Bad

  • GMV is based on gross, which is dangerous. If your margins are small, or negative, the value of GMV may vastly overstate the size of your business.
  • Speaking of margins, GMV does not penalize spending to generate more volume. Since it disregards margins, you can run negative margins and grow your GMV quickly at the cost of a viable business model.
  • Used on its own, GMV can be misleading as it is not a robust statistic. It needs to be combined with measures of central tendency and other statistics.

Personally, I prefer Net Market Value, which is the total net value of all goods or services sold, as the Gross can be misleading depending on your margins (which we’ve discussed previously). Still, understanding the total size of your marketplace, which is what GMV provides, is critical to understanding your performance.

“Advertising is legalized lying.” 

Advanced KPIs: Mobile Games – DAU/MAU

This is part 4 of a 5 part series on Advanced KPIs.

One of the key challenges in mobile gaming is retaining users in a world where there are so many games competing for their attention. When engagement and retention are your critical challenges, you need a KPI that captures the essence of both. All games track the total number of unique users active everyday, known as Daily Active Users, and the total number of unique users active in a given month, known as Monthly Active Users, but alone neither capture the frequency of use or retention of users.

To that end, many mobile gaming companies divide their average Daily Active Users (DAUs) by their Monthly Active Users (MAUs) to create a KPI called DAU/MAU.

Yes, that’s really what it’s called.

So why is that ratio a useful measurement? It captures a number of different factors in engagement at the same time:

  • The higher the ratio, the more of your users that are using your application everyday. A value of 0.5 or 50% means your users are playing your game roughly 15 out of the 30 days in a given month.
  • It captures customer churn in a basic way. If you are gaining a lot of users but losing them quickly, your DAU/MAU ratio will go down.
  • If you track it by cohort, you can see your customer retention over time as your ratio changes. For example, if your ratio is 0.5 the first month but 0.1 the second month, that means usages has dropped from every other day to only 3 out of 30 days in a month.

In recent years, as analytics tools have gotten more advanced, use of the DAU/MAU ratio has fallen in favor of more detailed metrics and tracking. While DAU/MAU is not used as widely as it once was, it has some surprising predictive characteristics in particular environments. For example, in the early days of the Facebook platform and the Apple AppStore, the DAU/MAU ratio was a fantastic predictor of success.

No KPI is perfect, let’s break down the strengths and weaknesses of this one.

The Good

  • It’s a very simple metric that you can calculate for every customer segment and cohort every month. This allows you to look at engagement and retention across different groups of users and identify problems through comparisons.
  • It’s widely known so there are many public benchmarks you can use to compare your performance to other games.

The Bad

  • Since it considers only Daily and Monthly activity, any user lifecycles in between are lost. For example, if you have a game that users love but can beat in a week, that will not be captured by your DAU/MAU ratio.
  • Since you need at least a month of activity data to determine your denominator, it cannot help you in the first month after your launch. This is unfortunate since many games will fail in that first month.

I will confess that I was originally very skeptical of the DAU/MAU ratio because of how simplistic it seems on the surface. Today I see it as a useful tool as long as it’s used in conjunction with other metrics like Lifetime Value (LTV) and Average User Tenure.

“Once the game is over, the King and the pawn go back in the same box.”

Advanced KPIs: Retail – Same-Store Sales

This is part 5 of a 5 part series on Advanced KPIs.

Retail stores have lost a lot of their shine in the past few years as the rise of e-commerce has reduced foot traffic in malls and shopping centers. Even so, physical retail stores represent the majority of sales worldwide and are a $4.8 trillion market in the US. With that much money flowing, the more data you have the better.

One metric that retail stores (and the analysts that follow them) use to measure the health of their business is Same-Store Sales. Same-Store Sales are the difference in revenue generated by a set of existing stores over a period of time from the sales in those same stores in the same period of time a year ago.

For example, if we have 10 sporting goods stores which sold a total of $50,000 in August of this year and sold a total of $45,000 in August of last year the Same-Store Sales for our business would be $5,000. Often when large retailers report on Same-Store Sales it is in comparison to revenue, so we might report our same store sales as 11.1% growth ($5,000/$45,000) in Same-Store Sales instead of $5,000.

Right, so what?

Why calculate same-store sales? Retailers are constantly opening and closing stores to try and improve profitability. With store locations changing so often it can be hard to track the core performance of a retail business. For example, a large retailer might open a large number of new stores to make their Gross Revenue look better while the amount they sell in each store actually goes down.

By focusing only on stores that have been open for a while and comparing their historical performance, Same-Store Sales is a more stable indicator of retail performance.

Let’s break down the strengths and weaknesses of this metric.

The Good

  • It’s very easy to calculate, so you can compute Same-Store Sales weekly, monthly and annually for any subset of the stores a retailer owns (by zip code, state, etc). This makes it a great tactical metric when calculated weekly and a great strategic metric when calculated yearly.
  • Retailers can use it as a performance metric for store managers, by comparing Same-Store Sales of locations with similar locations and volume.
  • Public market analysts love this metric as the percentage change in same store sales makes it easy to compare the health of different retailers regardless of size or industry.

The Bad

  • Even though a store might be in the same location as it was last year, it might be selling different products. It can be hard to take into account the shift in products and prices using such a simple metric and you might be comparing performance when it doesn’t make sense.
  • By focusing on Gross, Same-Store Sales ignore that the costs of operating stores might be changing over time. If wages go up in a certain location, or if the cost of supply a store spikes it would not be reflected in most Same-Store Sales metrics.

As you’ve seen this week, no metric is perfect! We can pull apart even the most common metrics used by specific industries to measure their performance. I hope you apply the same level of critical thinking to your own metrics, since it will ensure you make better decisions with them.

“Whoever said that money can’t buy happiness, simply didn’t know where to go shopping” 

Sign up for a single idea in your inbox every Monday, to help you make better decisions using data.

Share this Post