It's a fascinating column to read, especially now – five years on – at a time when 'data journalism' has taken root, and news media of all stripes are embracing the idea of telling stories through statistics: weaving together data and narrative to provide the sort of context that, Paulos might argue, is the preserve of the storyteller alone rather than some sort of statistically informed hybrid.
Paulos is right, though, that statistics deal better in averages and movements, rather than individuals and motives. This can be seen in our August 2015 cover story, which investigates whether the executions of soldiers for the crime of desertion during the First World War acted as an effective deterrent to others. The statistics can indicate whether or not there was a negative correlation between the number of executions and the number of convictions for desertion the following month, but they tell us nothing about those found guilty of the crime, their motives for fleeing and how they weighed the threat of execution against the fear of facing death in combat.
Similarly, in looking to tell a different story of human behaviour – this one about sex – David Spiegelhalter had to rely on aggregates and averages rather than individual accounts. His new book, Sex by Numbers, uses an assortment of statistics to explore trends over the past century and beyond. But it is not only a story of our private, intimate lives – the book also serves as a guide for how to think critically about statistics, for not all statistics are created equal, as Spiegelhalter explains in an extract.
While most people recognise that there are 'good' and 'bad' statistics, it is sometimes difficult to tell which is which. In his column, Paulos wrote that: 'In listening to stories we tend to suspend disbelief in order to be entertained, whereas in evaluating statistics we generally have an opposite inclination to suspend belief in order not to be beguiled.' It does not always work that way, however.
Sometimes our statistical reasoning is faulty, as William Skorupski and Howard Wainer explain, in the case of the prosecutor's fallacy: where a 3-million-to-one chance of a random DNA match becomes a 3-million-to-one chance that a person is innocent.
At other times, the fault is in the way data has been analysed and presented, which can inadvertently lead us to the wrong conclusions, as H. James Norton and George Divine discuss in their article on Simpson's paradox.
For nearly 12 years, Significance has been walking the line where statistics and storytelling meet. There is tension, certainly, but very little conflict. And I like to think that feedback from readers keeps the balance in check.
- Get six issues of Significance each year. The print magazine is available by subscription or as a member benefit in joining the American Statistical Association, or the Royal Statistical Society.
