4 ways to misinterpret your measurement

Our favorite is: “Data are like political prisoners: Torture them long enough and they’ll say anything.”

Now, we’re all for deep data analyses, and there’s nothing we love more than cross tabs. But unfortunately, as we know, torture doesn’t always lead to truth. It’s no different with manipulating measurement data. There’s a fine line between data analysis and misinterpretation.

Here are four ways you may be mishandling your data:

1. You’re asked for percentages of small subsets of all the data.

We once delivered a launch report to a client that included data from about 250 social and traditional media items. It showed that the desired messages appeared in about 7 percent of the coverage, which isn’t bad.

But it wasn’t what the client expected or wanted.

The client asked us to redo the analysis after eliminating all social media, arguing their outreach focused on traditional media. That reduced the data by half. It was still an acceptable amount of data to analyze under the circumstances and, as it happened, the data didn’t change that much. The percentage of items containing one or more key messages went up slightly to 10 percent.