Survey Design: Dimensional Analysis

This is part 5 of our series on Survey Design, previous segments are available in our archives.

After you’ve designed your questions, gathered your results and accounted for bias you have your survey data! However, if you stop there you are missing out on the real power of survey data which is dimensional analysis.

The great thing about survey responses is that, compared to most of your data, you can directly attribute each answer to a specific customer. This means that you can use the answers to one question to analyze and segment the answers to other questions!

For example, in the Data Driven Survey we asked you “How large is your company?”, “What industry does your company work in?” and “How often do you meet with your team to review your metrics and goals?” We can combine these answers to learn a lot about how different kinds of companies use data!

First, here is an analysis of frequency of metrics review by company size:


As you can see, regardless of company size the most common frequency is weekly meetings. Note that not all company sizes go to 100% as not everyone answered all of these questions (see yesterday’s discussion of bias).

When we look at the same frequency of metrics review by industry we see some big differences:


As you can see, consumer businesses (E-Commerce, Consumer Mobile) are much more likely to meet daily while Finance businesses are more likely to meet weekly or monthly. I only included the industries where we had enough responses for the data.

If we had enough responses, we could go to the next level and look at how the frequency of metrics reviews vary by company size AND industry. In fact, as long as you have enough data there is no limit to the multidimensional analysis you can do.

How do you do this kind of analysis with your survey data? The easiest way is to use a Pivot Table, which is available in most spreadsheet software. Pivot Tables can automatically create cross-tabulations which combine dimensions into results like those you see here. In fact, I created these charts using a Pivot Table in Microsoft Excel.

As you’ve seen this week, surveys can be a powerful tool in understanding your customers if you design them well and are careful in analyzing your data. Thanks again for participating in the Data Driven Survey, not only did the data make this week possible but it is helping us build Outlier, a product that will help you make better data driven decisions!


Quote of the Day: “In the long run, we are all three-dimensional side-effects of a two-dimensional universe existing in a multidimensional stack.” – Warren Ellis, Planetary