Security BSides, San Francisco | April 20, 2015

Slides | Video

Abstract:
Stiff statistics, prismatic pie charts, and stodgy survey results drown the Information Security space in a sea of never-ending numbers that can be difficult to sift through and find the relevant information contained within. Have you ever finished reading a vendor whitepaper or a research institution’s annual security report and felt your Spidey sense begin to tingle with doubt or disbelief? What you are probably sensing is a manipulation of statistics, an age-old hoodwink that has been occurring as long as numbers have been used to convey information.

This critical subject was first examined over 60 years ago, when Darrell Huff first published the groundbreaking book “How to Lie with Statistics,” over 60 years ago, and since then has become required reading in many college Statistics classes. This presentation takes the foundation Huff created and updates the core concepts for the contemporary Information Security field.

Most people would be shocked to find that data can be easily manipulated to leave the reader with a certain impression or to lead them to a particular conclusion. Nothing is sacred in this presentation! Several areas are examined, from bias in vendor-sponsored security reports to common ways pie charts are used to misrepresent data. Extra time is given to the scourge of risk analysts everywhere: the post hoc fallacy (correlation does not imply causation), perhaps the most prevalent and most damaging of all logical fallacies seen in Information Security.

There is a silver lining – once you are aware of the subtle ways data is manipulated, it’s easy to spot. Attendees will walk away with a new understanding of ways to identify and avoid unintentionally using some of the methods described.