Crime surveys – trust the data (because what else is there?)

September 3, 2007

The 2007 British Crime Survey is a document of quite staggering un-newsworthiness. 47,000 people were quizzed about their experience of crime in the past year, the data was collated and the results extrapolated to the nation as a whole. The upshot? In 2006 crime was largely stable, with small rises and falls in some categories, most of which were not statistically significant.

So naturally the media went crazy. The Times talked of “a million attacks by drunken thugs”. The Daily Express said “Street attack every 12 seconds as crime soars under Labour”. Almost every other paper was a variation on the same theme. Amidst the indignation, David Davis, the Shadow Home Secretary, accused the government of a “serial failure to protect the public.”Here are the summary figures: vandalism rose by 10 per cent, burglary fell by one per cent, violent crime rose by five per cent and overall crime rose by three per cent. The statisticians were quoted as saying the rises cherry-picked on the news pages – mostly violent crime – were (very much in quotation marks) “not statistically significant”. If ever punctuation could convey disdain, it does so with those quotes. As Norman Brennan, director of the Victims of Crime Trust, said somewhat enigmatically in the Express: “Five per cent may not seem a lot to some people but when you compare it against unprecedented rises in violent crime then it is very worrying.” Read that sentence a few times, and see if you can work it out.

In fact, if they had been asked, the statisticians would also have said that most of the falls recorded were also not significant. Because no one is suggesting that a rise in violent crime by five per cent is not worrying, what they are actually saying is that since the estimate was based on survey data, the rise was too small to confidently extrapolate to the population. That’s not reducing misery to mere statistics, it’s just the mathematical truth.

 What can be inferred though, using the same survey that all the media gleefully reported, is that violent crime has plummeted since 1995. Oddly enough, that never gets in. Here is the graph, broken down by type (remember, population has also risen since the data begins in 1981).   

Violent Crime

To which there is an obvious response, attack the survey. The most common arguments are below:

  1. Ah, but the police are so ineffective no one reports crime anymore
  2. The government are busy redefining crimes to massage the data

The answer to both of these is the same – the British Crime Survey does not deal in recorded police data, it simply asks a large set of people the same set of questions each year, then extrapolates the responses. The alternative, using recorded crime, would make longitudinal studies impossible – a redefinition of (say) pub fights as non-violent crime could create a hugely impressive discontinuity in any graph of crime trends.

The BCS is not without its problems though. A Telegraph correspondent accused it of “jiggery-pokery” for not recording “crimes which are serious but too small statistically to measure (such as murder and rape) or crimes committed against businesses (such as fraud or shoplifting) or against people under 16 (since it only surveys adults)”.

Leaving aside the issue of how precisely you conduct a survey chatting to murder victims, the correspondent is of course absolutely right. Perhaps a better survey could be designed.

But for the moment it is not only the best we have, it is pretty much all we have. And I would take that over a journalist’s intuition any time, particularly when papers seem to have a pathological desire to believe the worst of Britain.

Link to survey – so you can judge the data for yourself –


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s