Why Overlapping Confidence Intervals mean Nothing about Statistical Significance | by Prasanna Parasurama | Towards Data Science

“The confidence intervals of the two groups overlap, hence the difference is not statistically significant” — A lot of People The statement above is wrong. Overlapping confidence intervals/error bars say nothing about statistical significance. Yet, many make the mistake of inferring a lack of statistical significance. Likely because the inverse — non-overlapping confidence intervals — implies statistical significance. I’ve made this mistake. I think part of the reason it is so pervasive is that it is often not explained why you cannot compare overlapping confidence intervals. I’ll take a stab at explaining this in this post in an intuitive way. HINT: It has to do with how we keep track of errors. From this setup, the same people quoted at the beginning will erroneously infer that because the 95% CIs are overlapping, there is no statistically significant difference in age (at the 0.05 level) between groups, which may or may not be correct. As it turns out the difference is statistically

“The confidence intervals of the two groups overlap, hence the difference is not statistically significant” — A lot of People The statement above is wrong. Overlapping confidence intervals/error bars say nothing about statistical significance. Yet, many make the mistake of inferring a lack of statistical significance. Likely because the inverse — non-overlapping confidence intervals — implies statistical significance. I’ve made this mistake. I think part of the reason it is so pervasive is that it is often not explained why you cannot compare overlapping confidence intervals. I’ll take a stab