Statistics, Risk Analysis, and Misunderstandings

I married a Statistician, so this article sums the lectures I receive on a daily basis. Risk Management is statistical analysis, and I'm not sure how many folks in IT Security have Graduate level Stat exposure. So, the understanding of our statistical shortcomings is key. You need to read that entire article, twice. This statement struck me, as I've noticed a scary trend in IT Security:

"People who know a little bit of statistics - enough to use statistical techniques, not enough to understand why or how they work - often end up horribly misusing them. Statistical tests are complicated mathematical techniques, and to work, they tend to make numerous assumptions. The problem is that if those assumptions are not valid, most statistical tests do not cleanly fail and produce obviously false results."

As we outsource more security, and buy more products, we must be careful, as this statement is also true:

"People who know a little bit of IT Security - enough to use an IDS or SIEM, not enough to understand why or how they work - often end up horribly misusing them. Security tools use complicated technical techniques, and to work, they tend to make numerous assumptions. The problem is that if those assumptions are not valid, most security tools do not cleanly fail and produce obviously false results."

My wife's constant guidance in Statistics has been invaluable to my evaluations of IT Security Policy and Implementation. When I came across this article thanks to @alexhutton, I had to share it!



comments powered by Disqus