Three Dangerous Statistical Mistakes

Page 1 of 3

Published on Quality Digest (http://www.qualitydigest.com)

Home > Content

Three Dangerous Statistical Mistakes

By: Eston Martz Hint: Theyre not about adding and subtracting Its all too easy to make mistakes involving statistics. Statistical software can remove a lot of the difficulty surrounding statistical calculation, reducing the risk of mathematical errors, but correctly interpreting the results of an analysis can be even more challenging.
ADVERTISEMENT

A few years ago, Minitab trainers compiled a list of common statistical mistakes, the ones they encountered repeatedly. Being somewhat math-phobic myself, I expected these mistakes would be primarily mathematical. I was wrong: Every mistake on their list involved either the incorrect interpretation of the results of an analysis, or a design flaw that made meaningful analysis impossible. Here are three of their most commonly observed mistakes that involve drawing an incorrect conclusion from the results of analysis. (Im sorry to say that, yes, I have made all three of these mistakes at least once.) Mistake No. 1: Not distinguishing between statistical significance and practical significance Its important to remember that using statistics, we can find a statistically significant difference that has no discernible effect in the real world. In other words, just because a difference exists doesnt make the difference important. And you can waste a lot of time and money trying to correct a statistically significant difference that doesn't matter.

http://www.qualitydigest.com/print/17002

12/19/2011

Three Dangerous Statistical Mistakes

Page 2 of 3

Lets say you love Tastee-Os cereal. The factory that makes them weighs every cereal box at the end of the filling line, using an automated measuring system. Say that 18,000 boxes are filled per shift, with a target fill weight of 360 grams and a standard deviation of 2.5 grams. Using statistics, the factory can detect a shift of 0.06 grams in the mean fill weight 90 percent of the time. But just because that 0.06 gram shift is statistically significant doesnt mean its practically significant. A 0.06 gram difference probably amounts to two or three Tastee-Os not enough to make you, the customer, notice or care. In most hypothesis tests, we know that the null hypothesis is not exactly true. In this case, we dont expect the mean fill weight to be precisely 360 grams; we are just trying to see if there is a meaningful difference. Instead of a hypothesis test, the cereal maker could use a confidence interval to see how large the difference might be and decide if action is needed. Mistake No. 2: Stating that youve proved the null hypothesis In a hypothesis test, you pose a null hypothesis (H0) and an alternative hypothesis (H1). Then you collect data, analyze them, and use statistics to assess whether or not the data support the alternative hypothesis. A p-value above 0.05 indicates there is not enough evidence to conclude H1 at the 95-percent confidence level. In other words, we can reject the alternative hypothesis, but the null hypothesis may or may not be true. For example, we could flip a fair coin three times and test: H0: Proportion of heads = 0.40 H1: Proportion of heads 0.40 In this case, we are guaranteed to get a p-value higher than 0.05. Therefore we cannot conclude H1. But not being able to conclude H1 doesnt prove that H0 is correct or true. This is why we say we fail to reject the null hypothesis [1], rather than we accept the null hypothesis. Mistake No. 3: Assuming correlation = causation Simply put, correlation is a linear association between two variables. For example, a houses size and its price tend to be highly correlated: larger houses have higher prices, while smaller houses have lower prices. But while its tempting to observe the linear relationship between two variables and conclude that a change in one is causing a change in the other, thats not necessarily so: Statistical evidence of correlation is not evidence of causation. Consider this example: Data analysis has shown a strong correlation between ice cream sales and murder rates. When ice cream sales are low, the murder rate is low. When ice cream sales are high, the murder rate is high. So could we conclude that ice cream sales lead to murder? Or vice versa? Of course not! This is a perfect example of correlation not equaling causation. Yes, the murder rate and ice cream sales are correlated. During the summer months, both are high. During the winter months, both are low. So when you think beyond the correlation, the data suggest not that the murder rate and ice cream sales affect each other, but rather that both are affected by another factor: the weather. If youve ever misinterpreted the significance of a correlation between variables, at least youve got company: The media is rife with examples of news stories that equate correlation and causationespecially when it comes to the effects of diet, exercise, chemicals, and other

http://www.qualitydigest.com/print/17002

12/19/2011

Three Dangerous Statistical Mistakes

Page 3 of 3

factors on our health. Have you ever jumped to the wrong conclusion after looking at statistics?

About The Author

Eston Martz: Some people take to statistics and data analysis naturally. Theyre attracted to numbers and arent intimidated by formulas full of arcane symbols drawn from long-dormant languages. I gravitate to words, not numbers. In school statistics stupefied me. I feared it. And thats a shame, because analyzing data is an extremely powerful tool to help us understand the worldwhich is why statistics is central to quality improvement methods like lean Six Sigma. While working as a writer, I began to appreciate the beauty in a robust, thorough analysis. I wanted to learn more. To the astonishment of my friends, I started a masters degree in applied statistics. And since I joined Minitab, Ive learned that a lot of people feel the same way about statistics as I used to. Thats why I write for Minitabs blog: Ive overcome the fear of statistics and acquired a real passion for it. And if I can learn to understand and apply statistics, so can you. Quality Insider Article Six Sigma Article 2011 Quality Digest Magazine. All Rights Reserved. Home the Insider Videos Magazine Privacy Policy Resources Search Subscribe Advertise About Us
Source URL (retrieved on 12/18/2011): http://www.qualitydigest.com/inside/quality-insider-article/threedangerous-statistical-mistakes.html Links: [1] http://www.minitab.com/uploadedFiles/Shared_Resources/Documents/Articles/not_accepting_null_hypothesis.pdf

Twitter Ed

http://www.qualitydigest.com/print/17002

12/19/2011