Product
Outlier is automated data insights for your entire business.
Book a Demo
Industries
Outlier is automated data insights for your entire business.
Book a Demo
Solutions
Outlier is automated data insights for your entire business.
Book a Demo
Resources
Outlier is automated data insights for your entire business.
Book a Demo

Metric Translation

Metric Translation

This is part 1 of a 5 part series on Metric Translation.

The best policy when communicating using data is to let the data tell the story for you (see Data Storytelling). But what do you do when the story the data is telling is not obvious? How can you translate from the language of mathematics into a language your team can understand?

Take, for example the following data:

There have clearly been some jumps and increases recently, but what else is it hiding? Those cyclical patterns on the right, did they exist earlier in the data? It looks flat, but perhaps that is just because the numbers are too small to see.

Translating metrics is an art form, as there is no right answer when it comes to how to communicate them. Given any single metric (or group of metrics), there are dozens of ways you can translate it to make it easier to understand. Some are simple, like ratios, that translate page views and purchases into conversion rates, for example, while others are more complicated.

This week, we’ll review some common ways to translate your metrics in ways that might make the insights that come from them easier to communicate.

Tomorrow we’ll get started with how to use translations to make sense of rapidly changing data using logarithmic scales.

“Translation is at best an echo.” 

Metric Translation: Logarithmic Scale

This is part 2 of a 5 part series on Metric Translation.

If your business is doing well, you might see very rapid growth in specific metrics. For example, the following is the revenue for a newly launched product line for a fictional e-commerce company:

The rapid growth at the end makes the rest of the chart impossible to see. Using the actual value of the metric makes little sense in this case since the last values are so much larger than the rest!

One way to better visualize data with such different magnitudes is using a logarithmic scale. In such a scale, instead of having the evenly spaced y-axis labels progress using counting numbers (1, 2, 3… ), the labels are powers of 10 (10⁰, 10¹, 10², 10³…). The result is that you can see movement in a metric that changes in significantly in magnitude over time. Below is the same data as above, but now on a logarithmic scale:

It’s now apparent that the cyclical patterns in the later data are present in the earlier data, meaning that the pattern has continued even as the magnitude has increased. This kind of detail is hidden on the actual value chart since the early data is so small, but can be seen on this logarithmic view.

Exponential growth is often talked about in businesses, because if that is happening then you are doing well. One of the added advantages of a logarithmic scale is that you can easily judge if a metric is growing exponentially, since an exponentially growing metric will appear as a straight line on a logarithmic scale. For example, it is hard to tell if the following data is growing exponentially:

But on a logarithmic scale it forms a straight line, showing that is is exponential:

Of course, if the people viewing your data don’t understand that the scale is logarithmic these charts can be more confusing than useful. It is critical to label your charts in a way that the scale is unavoidable to avoid such confusion.

Tomorrow we’ll look at a more complex scenario, when you need to translate insights including more than one metric.

“Such is our pride, our folly, or our fate, That few but such as cannot write, translate.”

Metric Translation: Relative Change

This is part 3 of a 5 part series on Metric Translation.

While logarithmic scales are a useful tool for visualizing metrics that change significantly over time, how can you translate multiple metrics that are on vastly different scales to make them easier to understand? For example, let us try to compare the following two metrics:

It’s very hard to see any kind of relationship between those metrics, because the magnitude of their values is so different. Your first thought might be to use different axes for each series (one on the left and one on the right):

But there are a lot of problems with this approach, which we’ve gone into some depth previously in our series on Data Visualization. Generally speaking, dual-axes plots are a bad idea and should be avoided.

But, even if dual-axes plots were a good idea (which they are not), what would you do if you have more than two metrics? If you are primarily interested in how the metrics are changing you can use the relative change to visualize the relationship. For example, given these three metrics:

Instead of charting the actual value of all metrics, we can chart the percentage change of each metric, relative to itself, from a point in time. This translates all metrics from real values into percentages (change since the chosen point in time) and makes it easy to see how they are changing together:

From this you can see that the changes in Metric 1 are significant, while the changes for Metric 3 and Metric 2 are smaller. This significant difference is not visible in the raw data.

Again, if the reader does not understand these are relative changes and not actual metric values there can be a lot of misunderstanding. However, with the appropriate context to avoid that confusion, you can use relative change charts to convey a variety of very interesting insights.

Tomorrow we’ll go a step farther and discuss how we can use residuals to emphasize the most interesting aspects of our data.

“There is not a fragment in all nature, for every relative fragment of one thing is a full harmonious unit in itself.” 

Metric Translation: Residuals

This is part 4 of a 5 part series on Metric Translation.

A residual is the difference between the actual data values and the trendline (see Trendlines) that you think represents the nature of the data. We have covered residuals previously (see Anomalies) as a useful tool for visualizing changes in data that might not be obvious from the raw data. Another way to think about residuals is a more advanced form of the relative change translation we covered yesterday.

For example, consider these two metrics which are changing over time:

There are some blips and trends, but it’s unclear how the two metrics are moving in relation to each other because the scale is so different. Instead of using a relative change chart, let us try charting just the residuals for the two metrics. For a trendline, I’ll use a 7-day moving average (see Moving Average Trendlines).

The residuals of both metrics tell an interesting story:

Both metrics changed significantly on 2/12/17, as indicated by the significant dip in the middle! This change was almost entirely hidden by the raw data, but is clear when looking at the residuals.

Translating multiple metrics into their residuals help identify these kinds of changes because you’re comparing the metrics to themselves, so any deviation from their typical behavior will become clear. Note that it is almost impossible to understand the nature of the actual data from the residual, so you should accompany any chart of a residual with the actual data it is derived from.

There are dozens of other translations we could cover this week, but tomorrow we will end with a warning about what happens when you take metric translation too far.

“Luck is the residue of design.” 

Metric Translation: Translations Gone Wrong

This is part 5 of a 5 part series on Metric Translation.

One of the great dangers in translating your metrics is that the original meaning gets lost along the way. The more you manipulate the data, the more likely you can introduce new meanings or even confuse the original insight by adding bias.

For example, consider the following metric that is growing fairly steadily over time:

There does not seem to be any clear signal in that data, so we might be tempted to translate it into another form. Here is what happens when I go overboard and make two translations, the first being the relative change and the second plotting it using a logarithmic scale:

I’ve managed to turn a steadily increasing chart into one that implies the metric is shrinking! If you use enough translations, you can make any metric tell any story you want.

Even translating the data with the best of intentions we can make it very misleading. There are many other ways to do this (see Numbers Lie), so you need to be careful and use your judgement when applying metrics translations. While there are no steadfast rules on translations, here are some guidelines:

  • Use the fewest translations possible. As you can see from the example above, the more translations you apply, the easier it is to mislead.
  • Simpler translations are betterThis week we’ve gone from simple (log scales) to complex (residual) translations. The simpler the translation, the easier it will be for your audience to understand.
  • Sanity checkLook at the final product and ask yourself how it could be misinterpreted. Will someone seeing this for the first time see the same meaning that you know exists?

Translations are a powerful tool, but like any powerful tool they are dangerous if mishandled!

In Review: While the data should speak for itself, often you need to translate it to make it accessible to the rest of your team. There are many methods to do so, ranging from different scales to calculations and each carries the promise of making insights clear and the danger of hiding the real meaning.

“Poetry is what is lost in translation” 


Sign up for a single idea in your inbox every Monday, to help you make better decisions using data.

Share this Post