Numbers Lie: Interpretation Bias

This is part 3 of our series on Metric Bias, previous segments are available in our archives.

What do you want to see?

One of the great challenges of making data driven decisions is letting the data shape your opinion. It is human nature to seek out data that reinforces our existing ideas and conclusions, something psychologists call Confirmation Bias. But using data to justify existing decisions is NOT data driven, because the decision was already made!

When you look at data and see what you want to see, I call it Interpretation Bias and it’s easy to do when your data sets have no clear message. Let’s assume the following chart is your corporate revenue and your boss asks you to determine if the business is growing:

IntBias

This data is very inconclusive, as is most real world data. However, because it is inconclusive you can read into it whatever you want! In fact, I’ve recorded a short video (3:51) that shows how easy it is to misinterpret this chart and goes into Interpretation Bias in more detail. Give it a watch if you’d like to learn more.

To avoid Interpretation Bias, there are a few simple steps you can take:

  • Self Awareness. Before even looking at data, admit to yourself what you want the data to say. Then do your best to see the opposite in the data, taking a devil’s advocate approach. It will force you to see the data from the other side at first.
  • Peer Review. Have someone else look at your conclusions and verify they reflect the data. If possible, make sure that person has no direct incentive related to the conclusion!  
  • Triangulate. While a single source of data might easily fall victim to Interpretation Bias, it is much harder when you combine multiple data sources together.

 

Quote of the Day: “It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.” – Sir Arthur Conan Doyle