Numbers in a spin

Tom Whipple
294 words
19 December 2008
The Times
1
5
English
(c) 2008 Times Newspapers Limited. All rights reserved

Most of the time the Government manifests its disdain for the awkward rigours of statistics by cherry-picking positive data or rubbishing negative data. Sometimes, however, circumstances call for a more spectacular gesture. This month the Royal Statistical Society achieved a small victory in ensuring that statistics remain independent from politics.

As of December 1, the Government is forbidden from receiving pre-release copies of statistical publications more than 24 hours in advance — previously it had five days’ notice. It’s probably not a matter of life and death, and it doesn’t go far enough, but this is a good thing. Statistics should be like the weather: irrefutable facts about the world outside. Give a Government spin-doctor a five-day start and anything can happen. Suddenly the sky is always blue and winter storms come from a butterfly flapping its wings in an opposition constituency.

So what happened? Did the Government sneak a report out 25 hours in advance? Did it hold a succession of five-day nostalgic pre-release parties starting on November 25? No. It got its hands on, and released, statistics that were not due for publication for three months — from a report that was not even completed.

The UK Statistics Authority has rarely been so strident. Jacqui Smith’s release of figures showing an apparent fall in some hospital admissions linked to knife crime was “premature”. The data were “irregular and selective”. The Home Secretary apologised but no one explained how the report was received.

Actions such as this undermine statisticians and destroy people’s faith in numbers. And, when it comes to evaluating knife crime policy, that may just be a matter of life and death after all.

Turning to God

Tom Whipple
377 words
12 December 2008
The Times
1
5
English
(c) 2008 Times Newspapers Limited. All rights reserved

A newspaper’s relationship to trends is much like a crank clairvoyant’s relationship to ghosts. Any rattle or shake is grasped as proof, any conflicting evidence as argument that we’re not looking hard enough. But, in their more reflective moments, psychics and newspapers alike probably suspect that it is all nonsense.

While clairvoyants are interested in proving the presence of your long-deceased great-uncle, newspapers nowadays want to prove the omniscience of something far more tangible: the credit crunch. If there is a trend — even, or perhaps especially, if that trend exists only among the Editor, the Editor’s mate and the Editor’s mate’s cat — then it must be linked to the economy.

So it was that The Sunday Telegraph told us this week: “As we face the most frugal Christmas for years, churches are witnessing a steady rise in attendance.” In 1,874 colourful and impassioned words, the author talks about an emergence of new values: “A belief in the value of frugality, modesty and charity. An appreciation of the role of the family.

Even a dislike of vulgarity, perhaps.” How heartwarming. His evidence for this cultural shift? “A straw poll by this paper reveals that church attendance was up last Sunday, the first of Advent.” Then on Monday, perhaps encouraged by a clairvoyant, the journalism fairy struck and provided genuine statistics. “Thousands of people are turning to religion during the recession,” the Telegraph claimed again.

This time, though, it had data. “Back to Church Sunday, an annual service in September when church members invite a friend, attracted 37,000 new congregants this year, almost twice the number in 2007.” In newspaper terms, that’s not a trend — it’s a cast iron, copper-bottomed, bona fide social phenomenon.

Except Back to Church Sunday has been going for just four years, and in each of those years more churches have joined the programme. So what would be surprising would be if, in a year when the scheme was still growing, the scheme didn’t grow.

And still no one has explained what any of this has got to do with the credit crunch.

Sample minds

Tom Whipple
253 words
5 December 2008
The Times
Times2 5
English
(c) 2008 Times Newspapers Limited. All rights reserved

“Common sense,” Einstein said, “is the collection of prejudices acquired by the age of 18.” The Enlightenment was the victory of science over common sense. Common sense tells us that the Earth is flat and is circled by the Sun. The slow accumulation of data, powering the steamroller of science, tells us that both assumptions are false.

Maybe common sense needs rehabilitating. Earlier this month, the Daily Mail told us that “teenage boys believe using pressure and alcohol to get girls into bed is ‘acceptable'”.

The data came from a study of 35 14, 15 and 16-year-olds of both sexes who were asked, in small gender-segregated groups, to talk about attitudes to sexual relationships. They were selected from those visiting sexual health clinics, in areas with high teen pregnancy rates.

Imagine you are in a room, asking teenage boys about sex. Since they have all visited sexual health clinics in areas where teen sex is a problem, would you draw conclusions from them about the non-sexually active population? Would their answers be a story?

Last year the United Nations admitted that it had overestimated Aids prevalence in Africa. The reason? Sampling is difficult in Africa, so they took the only figures they had: HIV prevalence among women in antenatal clinics. So their sample population consisted entirely of women who had had unprotected sex.

Numbers are imperfect, because research is imperfect. Sometimes interpretation requires a little common sense.

As many as none

Tom Whipple
368 words
24 October 2008
The Times
Times2 5
English
(c) 2008 Times Newspapers Limited. All rights reserved

As many as three quarters of all Times journalists are, at any one time, drunk. Up to half of those have, on occasion, stolen a colleague’s press pass, staggered to the nearest station, and sold it to a tramp for half a can of White Lightning and a pinch of Cutter’s Choice.

On Friday afternoon, the home news editor leapt on to the foreign editor’s desk wearing a leopardskin thong, danced the rumba, and growled at passing reporters until they gave him beer money.

Only one of the above statements is false, but let me rewrite the first two. Between 0 and 75 per cent of all Times journalists are, at any one time, drunk. Less than 32.5 per cent of – and perhaps as few as zero – Times journalists have sold press passes for alcohol. This tells us nothing.

Where is this going? Last week, to choose a story at random, it was reported that a grim high street was preparing itself for recession. The evidence? Shops are cutting prices by up to 75 per cent. To test if an assertion is silly, try flipping it on its head: in this case, no shop is cutting prices by more than 75 per cent.

Anyone familiar with the television advertising oeuvre of DFS will know that there has not been a period in the past decade when a shop in Crittal’s Corner, New Malden or Milton Keynes has not cut the price of something by 75 per cent. So a single sofa could always have justified this “credit crunch” story.

A real story would be if average prices fell across all shops, but that would require research – and would also contradict the same paper’s assertion that things are getting worse for “hard-working families”.

“Up to” and “as many as” are phrases that are extremely useful for journalists. They can be used without evidence and they can be justified with only one example – or one upper-bound estimate by one scientist.

But on as many as 100 per cent of the occasions they are used, they are completely meaningless.

Absolute zeros

Tom Whipple
362 words
17 October 2008
The Times
Times2 7
English
(c) 2008 Times Newspapers Limited. All rights reserved

The figure Pounds 2 trillion is unimaginable. Human beings were simply not evolved to imagine numbers on the scale of the sums used this week to save the world’s banks.

That is why we use comparisons. For instance, with rainforest destruction, instead of talking in hectares, we use the internationally accepted unit of deforestation: Waleses per second (for which the symbol is Taff s-1). How, then, to describe Pounds 2,000,000,000,000? The trick is to use examples relevant to the subject – in this case, bankers.

Let us say that, in these credit-crunch days, God is feeling ironic and makes it rain Bollinger in the Thames catchment area. In these circumstances it will take six months before Pounds 2 trillion worth of bubbly flows past Westminster. Now imagine that, in a sly reference to some traders’ illegal vice of choice, He also produces a gentle snowfall of cocaine over the City – enough to cover the Square Mile so that nothing is visible beneath. For Pounds 2 trillion, we could have 13 such snowfalls.

Perhaps a similar sum should be spent boosting Iceland’s economy, in banker approved fashion. At going rates you could fly the country’s entire adult female population to London and pay them to dance (as long as they are compliant) continuously for eight years. To complete our capital’s transformation into a den of City debauchery, we need some bling. What better than another London Eye, but made of solid gold? Well, we can afford 160 of them.

Our bankers’ theme park ready, we should expect it to be popular. Let each visiting banker, wearing a Rolex watch, arrive in a Porsche. In the passenger seat his trophy wife can be holding a Pounds 5 million Faberge egg. Oh, and last year’s Pounds 10 million bonus (in the form of a small Picasso that matches the curtains) is in the boot. Before this traffic jam achieves a net worth of Pounds 2 trillion, it will reach from London to Glasgow.

Questionable answers

Tom Whipple
361 words
3 October 2008
The Times
Times2 7
English
(c) 2008 Times Newspapers Limited. All rights reserved

There are two explanations for the regular Daily Express phone poll. The first is that it is genuine, that every day it asks readers such questions as: “Is Brown our most untrustworthy PM ever?”, or “Does our benefit system reward scroungers?”, because it really wants to know that 98 per cent of readers agree. The second explanation is that, in a fiendishly clever satire on its readership, the paper is repeating the work done in 1993 by Kunda et al into the Acquiescence Effect.

Bias in questions is a fascinating subject, and that paper added to a body of evidence indicating that the most subtle changes in questions can have a profound effect. Asking the same yes/no question twice, but reversing the emphasis, yields different results. For instance, people are more likely to answer yes to “Should smoking be banned in public?” than answer no to “Should smoking be allowed in public?”

Enter the Express. Its phone poll has two important characteristics. The first is that, for an Express reader, there is an obvious right answer. When asked “Are you fed up with fanatics changing Britain?”, or “Do you agree? Is Brown appalling?”, 99 per cent say “yes”. The second is that, just occasionally, the right answer is “no” (as with the ever-controversial: “Is it fair to make us work until we drop?”).

Here is where it gets interesting. If the research is correct, we would expect fewer people to answer “no” when no is the right answer than answer “yes” when yes is the right answer. And that seems to be the case. In a random selection of 40 previous polls, 32 are “yes” polls, and the average result is 98 per cent; in the other eight, the average result for “no” is just over 95 per cent. Performing a statistical check on the data, the Mann-Whitney U Test, it turns out that less than once in a hundred would such a difference happen purely by chance. People really do prefer to answer “yes”. Which means that the Express can stop those silly polls now.

Topic of cancer

Tom Whipple
295 words
26 September 2008
The Times
Times2 7
English
(c) 2008 Times Newspapers Limited. All rights reserved

Five is a big number, 0.0005 per cent is a small number, and 5 x 0.0005 is still a small number – at least in the context of cancer.

The Radiation Research Trust, a pressure group campaigning against mobile phone masts and wi-fi, held a conference last week at which Dr Lennart Hardell, a Swedish researcher, claimed that mobile phones raise the risk of contracting certain brain cancers by five times. This was serious: as all the national papers that reported it pointed out, the conference was at the Royal Society. We had a credible expert, we had a big risk and we should be worried.

The study has not been published yet, but when contacted Hardell is happy to talk about his research. It seems to show that 20 to 30-year-olds who have used a mobile phone for ten years have a fivefold greater risk of developing glioma, a fatal brain cancer.

How many 20 to 30-year-olds develop glioma in Britain? About five in every million – seven times as many die from accidental poisoning. Any deaths are troubling, but multiply a small number by five and you still have a small number: 25 in a million. We are not, as some papers implied, heading for an epidemic. There is another problem. The rarer an event, the larger the sample needed to be sure of what is happening. As Hardell freely admits, his numbers are low – the research is at best cause for further investigation.

And the Royal Society? I phone its spokeswoman, who says: “We operate as a conference centre, as well as being the Royal Society. This organisation just hired one of our rooms.”

Iraqi bodies count

Tom Whipple
367 words
19 September 2008
The Times
Times2 5
English
(c) 2008 Times Newspapers Limited. All rights reserved

“We don’t do body counts”. So said General Tommy Franks in 2002, in response to a journalist’s question about Afghan civilian fatalities. Later, when President Bush declared “Mission accomplished” in Iraq, recording civilian deaths was still far from a priority.

No one was pretending it was mission accomplished when General David Petraeus presented his report on the progress in Iraq a year ago this month. Buried in the appendices was a reversal of General Franks’s statement as well: slide three was entitled “Iraq Civilian Deaths”. The Pentagon did do body counts after all.

The Iraqi fatality statistics, published quarterly, are now a central argument for the coalition. They show a fall from 3,700 monthly deaths in December 2006 to around 500 today.

It was not always so. In the beginning a website, IraqBodyCount.org, was not just the standard measure for civilian deaths in Iraq – it was the only measure. Unlike the Pentagon, whose sources are unclear, IBC’s methods are simple: they count deaths recorded in the media, providing upper and lower bounds. They soon came to be relied upon by much of the world’s media.

Initially, the Right was furious. But nowadays most attacks on IBC come from the Left: it is accused of undercounting rather than overcounting.

In 2004 a study in The Lancet concluded that 100,000 people had died in Iraq. The same team estimated last year that the death toll had reached 655,000. The IBC records fewer than 90,000. And when, in 2005, President Bush was asked directly by a reporter how many civilians had died in the conflict, his answer of 30,000 was close to the estimate provided by IBC at the time. It seems that with a free market in body counts, Bush had shopped around for the most favourable statistics.

Even for neo-conservatives, outsourcing body counts is the limit of market liberalism. At some point, someone in the Pentagon started counting. Their statistics may be dismissed as propaganda, produced using unpublished methods, but – even as propaganda – civilians, and statistics, matter again. That’s no bad thing.

A percentage point

Tom Whipple
360 words
12 September 2008
The Times
Times2 5
English
(c) 2008 Times Newspapers Limited. All rights reserved

Shampoo adverts follow a comfortingly reliable formula. First, swishy models with swishy hair swish across the screen. Then “The Science Part” shows computer-generated nano-particles livening up dead roots and informs us that “68.18 per cent of Superhair hair product users had swishier hair”. In tiny print across the bottom are the words “15 out of 22 respondents noticed positive results”.

The great thing about percentages is that they make samples seem bigger, results more accurate. It all seems more – well – sciencey.

And these techniques don’t just apply to hair. We learnt in the Daily Mail this week that, since the introduction of a curfew for teenagers in Redruth, crime has “plunged” by 15 per cent. Excellent news. And how many crimes was that? Fifteen. In fact, the paper got its sums wrong: it said that crime had dropped from 121 incidents in August last year to 106 this year, a fall – using its own data – of 12 per cent.

Reporting on a study by the Equality and Human Rights Commission into the “concrete ceiling” faced by women, various newspapers told us this week that the proportion of women in the Cabinet has fallen from 34.8 per cent in 2006 to 26.1 per cent. How many is that? From eight to six. There is definitely cause for concern about equality in many professions – but would papers really have reported a drop of two people as significant if expressed in that way? Put it in percentages, though, especially with decimal points, and suddenly it has gravitas.

Percentages are great for simplifying changes in large numbers, or numbers that themselves involve decimal points, such as prices. But when the numbers involved are both small and whole, it is difficult to see why we need to convert them to percentages. All that it achieves is to move the consumer of the statistics further from the raw data itself so that they are less able to evaluate its importance. Unless, of course, that is the point.

Your average storm…

180 words
3 September 2008
The Times
Times2 5
English
(c) 2008 Times Newspapers Limited. All rights reserved

The average person has one breast and one testicle. But, outside the darker imaginings of the internet, it is hard to find such a person. Averages are not always useful. As with body parts, so with rain. According to The Express’s Saturday front page, “10 days of rain”, 55mm, would fall last Sunday – quite a deluge. A natural assumption about averages is that they fall in the middle, but that is not true with rain. The distribution is asymmetric: the lowest amount of rain in a day is bounded by zero, while the highest is essentially unlimited. The Met Office says there are only 150 to 200 days of rain each year, so even on the average rainy day, two days of rain fall. In fact, the Met expects summer thunderstorms that exceed (for short periods) 100mm of rainfall an hour. Misunderstandings about averages put an average August storm on the front of a national paper.