People are not perfect, and neither is data, so inevitably your data or process will let you down. This can happen in many ways including:
- Data quality problems that lead to metrics being inaccurate or unreliable.
- Missing a significant insight in the data, which has a big impact on the business.
- Not tracking the right data and as a result being unable to answer certain questions.
Whatever happens, the end result is always the same: people lose faith in the data and start to fall back on their intuition and experience. It is hard to regain that trust, but very important to do so to ensure the future of the business.
Here are some important ways to regain lost trust in data:
- Post Mortem. A post mortem is a process where you review what went wrong, what the root causes were of the problem and how you can ensure it doesn’t happen again. Doing a post mortem and publishing the results to your team will show how seriously data issues will be taken and resolved. It will also help them understand what happened, and that understanding will help prove the problem was specific to this one case, not generalized to all data.
- Continuous Improvement. People will be more forgiving of problems if they see a constant stream of improvements in your data and data process. Don’t wait for problems to arise, make sure to publicize a constantly stream of improvements. When you are done with your post mortem, whatever fixes you have identified should fit into this process as part of your normal operations.
- Carry on. The worst thing you can do is stop all of your data activities, as consistency is the most important factor in surviving a crisis. Keep publishing metrics, training users on data and doing investigations. Your confidence in data will encourage confidence in others.
Your chances of quickly recovering trust in data will be directly related to whether you are the one who finds the problem or not. If you find it yourself, you can be proactive and handle the situation. If not, rumors and backchannels will undermine your ability to regain that trust.
To make sure that you can be proactive, here are some important things to do starting now:
- Data Quality Monitoring. Don’t wait until a problem arises to find a problem in your data, make sure you are monitoring data quality everyday. This can be in the form of automated testing, manual checking or simple cross-validation metrics, which will highlight major issues in your data. Whatever approach works for you, the most important part is to do is regularly.
- Build an Issue Reporting Process. Just like with software development, you need to set up a way for people to report data issues (or suspected issues) for investigation. You will likely get a lot of false reports of data issues, but you will gain insight into issues as they are first detected. Be sure to respond to all reports, even if they are not really problems, to build the confidence that all reported issues are reviewed.
I mentioned this earlier, but it’s critical to repeat: Your confidence in data will encourage confidence in others. Having confidence in your data and conveying that to your team is likely the most important thing you can do.
In Review: Building a culture of using data can be difficult, but as long as you start simple and focus on education and fundamentals you can be very successful. When you have reached success, data will be an invaluable part of company operations and it will pay for itself many times over.
Quote of the Day: “I’m not upset that you lied to me, I’m upset that from now on I can’t believe you.” ― Friedrich Nietzsche