Outlier is automated data insights for your entire business.
Book a Demo
Outlier is automated data insights for your entire business.
Book a Demo
Outlier is automated data insights for your entire business.
Book a Demo
Outlier is automated data insights for your entire business.
Book a Demo

Predicting the Future

Predicting the Future

This is part 1 of a 5 part series on Predicting the Future.

Predicting the future is hard. I am fairly certain of this because I’ve never won the Powerball and neither have you.

Still, you need to predict the future to make effective decisions. Knowing what has already happened (using data) is not enough, you need a model for what the future holds to decide how best to move into that future. Will demand for your product grow? Will competition drive down prices? Will your costs grow or shrink over time?

I have no doubt you can predict what you’ll be doing in an hour by checking your calendar. What about next week? Next month? Next year? The farther into the future you try to predict the harder it becomes. You need the help of both data and models to reliably predict what will happen far enough in advance that you can do something about it.

We’ll cover a number of different ways to predict the future for parts of your business and how it can give you an advantage in decision making over your competitors!

WARNING: predicting the future is not an accurate science. Unlike some of the other topics we have covered, it will never be 100% reliable! Even so, it can provide valuable guidance.

“The future will soon be a thing of the past.” 

Predicting the Future: Growth Modeling

This is part 2 of a 5 part series on Predicting the Future.

One of the most common questions we try to answer using data is what will our business look like next month? Next quarter? Next year? The better you understand what your business will look like in the future the better you can prepare for it, or change it!

But, predicting the future of your business is difficult because real world data is complex. For example, given the following daily revenue data let us try and predict the revenue for each day next week:

Like most real world data there are patterns and trends hidden in this data, making extrapolation difficult. Basic techniques like Linear Regression would give us a general trendline, but not a prediction for every day next week. We could use more advanced techniques like Double Exponential Smoothing but they are difficult to implement if you aren’t familiar with them.

Luckily, there is an easier way to model the growth of cyclical real world data. Instead of trying to understand the data as a whole, we can realize that the repeating cycle means we can focus on each day of the week independently. For example, if we just take the Mondays the trend and pattern is actually a straight line!

By creating a separate trend for each day of the week, we can build a model for what we think revenue will be every day next week with a fairly high degree of accuracy:

Awesome! But I don’t care about days of the week.

Very few businesses care about predicting revenue for each day next week. However, the same technique can work with weekly, monthly or even quarterly data that follows common cycles. This is true for many types of business, especially seasonal businesses like tourism. If your data doesn’t have clear cycles or trends, then you can fall back to either general forecasting with trend lines or more advanced techniques like mentioned above!

Pro tip: As with many of the things we cover this week, you’ll likely use tools to help you predict the future values of your metrics. My hope is that by understanding the techniques these tools rely on you will be a better customer when selecting them!

“Life is divided into three terms – that which was, which is, and which will be. Let us learn from the past to profit by the present, and from the present, to live better in the future.” 

Predicting the Future: Lead Scoring

This is part 3 of a 5 part series on Predicting the Future.

Yesterday we covered predicting how metrics for your business might change in the future. That helps at a high level, but what about predicting the outcomes of the activities you do everyday in your business?

One of the most common elements of any business are leads. These are potential customers who have either made contact with you or you have contacted them, but you haven’t yet started moving them through the sales process (or conversion funnel). The faster your business grows, the harder it will be to spend as much time with each lead as you would like. It would be great to have a technique to predict how likely a given lead is to convert into a customer so you can spend your time on those high potential leads!

There is and it’s called Lead Scoring.

Traditionally, Lead Scoring is a process that ranks all of your leads, ordering them from “hottest” to “coldest.” How does it work? It starts with data collection. Relevant data that helps score leads can include demographic information about the company / person to whom you are selling as well as their behavior metrics.

For example, useful company-level demographic data includes the company’s industry and size. Personal demographic information includes the person’s title and role at the company, particular does she or he have budget or authority to purchase what you are selling. Behavioral metrics include the number of times a person has opened an email from you or attended webinars that you offer.

After you’ve determined what you think are the best attributes that describe your leads, you can allocate points to each one, giving more points to the most important attributes (and you can even give negative points for “bad” behaviors, like unsubscribing from a mailing list). Once the points are tallied, sort your list in descending point value and you are left with the “hottest” leads at the top of the list. These are the leads that you should pursue now while others lower on the list need more cultivation.

How do I predict if the lead will convert?

Though the traditional scoring mechanism is helpful, it does not provide you a prediction as to whether the lead will convert or not. However, there are a number of machine learning techniques that can help you. One of the more basic approaches is something called a logistic regression. A logistic regression is similar to a linear regression except that the variable you are trying to predict is a binary outcome, will the customer convert or not in this case, as opposed to a continuous value.

No matter the technique, all of them will rely on high-quality historical data that is used to train your predictive model – the more of it the better! As you go score leads and process outcomes, you’ll want to continually re-evaluate which attributes you are collecting data, and how you weight your scoring, to see if you can better predict your outcomes in the future.

As with many of the things we cover this week, you’ll likely use tools to help you with lead scoring. My hope is that by understanding the techniques these tools rely on you will be a better customer when selecting them!

“A person often meets his destiny on the road he took to avoid it.”

Predicting the Future: Churn Prediction

This is part 4 of a 5 part series on Predicting the Future.

For most businesses, it’s much cheaper to retain existing customers than to acquire new customers. Part of that depends on your Customer Acquisition Cost, but a lot of it is due to existing customers being highly qualified.

So, if we don’t want to lose customers we should make them all happy, right?

Sure, but that is harder than it sounds. No product makes everyone equally happy, and there are always new competitors waiting in the wings trying to steal our customers away. It would be great if there was some way to know which customers were most at risk so we could focus on making them happier.

There is and it’s called Churn Prediction.

Churn Prediction works much like Lead Scoring, but instead of estimating the likelihood a lead converts into a customer it estimates the likelihood that a customer will churn. The same sort of company / personal demographic data we discussed yesterday (e.g., company size) are appropriate here. However, the behavioral data will need to be updated to also include information about how the customer has engaged with your product.

  • Product usage: How many times has the customer logged in? What is the cadence of their interaction with your product? How many users are there using your product at the customer? Has the customer changed their level of service (upgraded or downgraded)? What is the tenure of your relationship with the customer?
  • Support: How many support tickets have been filed by the customer? What was the severity of each ticket? What was the tone of the tickets? How quickly were the tickets resolved?
  • Social: Has the customer every mentioned you in social media? Has the customer referred you to any other companies? Are you allowed to use the customer’s logo on your website? Was the customer willing to produce a case study for you to publicize?

With all of these attributes defined and data collected, you’ll be able to score all of your customers and have a good sense as to which ones are likely going to stay customers and which are going to churn.

How do I predict if the customer will churn?

Like Lead Scoring, creating a score for churn does not provide you a prediction as to whether the customer will churn or not. The logistic regression technique I mentioned yesterday is still applicable here, but there are other options as well, such as the class of models called Markov Models. These types of models are state-based, in other words, you define the probability of moving between different states under a key assumption that the probability of moving from one state to another is only dependent on the last state you were in (the Markov assumption). The Hidden Markov Model makes the most sense because, as the name suggests, a customer’s commitment to your product is hidden, i.e., not directly observable. With a model of this type, you’ll be able to understand the paths customers take to renewal and churn.

As we discussed yesterday, with any modeling effort, data quality will strongly influence your results and you will want to continually evaluate the attributes and parameters of your model to better predict your outcomes in the future.

You’ll likely use tools to help you with Churn Prediction; I hope this post is helpful to get you more familiar with the techniques these tools rely on so you are better informed when making your choice.

“We see our customers as invited guests to a party, and we are the hosts. It’s our job every day to make every important aspect of the customer experience a little bit better.”

Predicting the Future: Market Predictions

This is part 5 of a 5 part series on Predicting the Future.

This week we’ve covered a lot of topics about predicting aspects of your business (both large and small). If only your business was entirely within your control, how easy life would be!

Alas, your business exists in a larger market and the changes in that market can have significant impacts into your business. Trying to predict the future without compensating for larger market trends would be like charting a course through the ocean and ignoring the weather.

Predicting the future of your market is harder than predicting aspects of your business because you will lack enough data to do it well. However, the goal of today’s topic is to encourage you to think about big-picture trends that can help you identify leading indicators of success for your company.

To lead or to lag

Many of the KPIs you follow are lagging indicators of success, like revenue. In other words, you can react to the KPI to help you think about what you can do better in the future. Leading indicators, on the other hand, help you be more proactive and change course before the final outcome. The discussion from yesterday around Churn Prediction is particularly relevant here; if you can figure out which customers are at risk of churning before they do, then you still have the potential to improve the relationship and retain the customer.

Customer churn is a fairly low-level leading indicator that will be common to pretty much any business. The challenge you face is finding higher-level macroeconomic leading indicators that can help you change course before it is too late. For example, if I worked at a sports team, I could analyze the number of HDTVs sold per capita to see if that is a leading indicator for in-person attendance to my events.

Strategic planning

As with everything in predictions, there is no crystal ball that will perfectly tell you what will happen in the future. The goal is to find the most reliable leading indicators of your business that will give you enough time to adjust your strategic plan before it is too late.

Next Week: I’ll toss Data Driven Daily back over to Dr. Doug Mitarotonda who will spend a few weeks talking about simple ways to analyze and visualize data, and explore the tools that you can use to tackle these tasks. In the meantime, have a great holiday weekend!

“I now have had my foggy crystal ball for quite a long time. Its predictions are invariably gloomy and usually correct, but I am quite used to that and they won’t keep me from giving you a few suggestions, even if it is merely an exercise in futility whose only effect is to make you feel guilty.”

Sign up for a single idea in your inbox every Monday, to help you make better decisions using data.

Share this Post