Questionable answers

June 2, 2009

Tom Whipple
361 words
3 October 2008
The Times
Times2 7
(c) 2008 Times Newspapers Limited. All rights reserved

There are two explanations for the regular Daily Express phone poll. The first is that it is genuine, that every day it asks readers such questions as: “Is Brown our most untrustworthy PM ever?”, or “Does our benefit system reward scroungers?”, because it really wants to know that 98 per cent of readers agree. The second explanation is that, in a fiendishly clever satire on its readership, the paper is repeating the work done in 1993 by Kunda et al into the Acquiescence Effect.

Bias in questions is a fascinating subject, and that paper added to a body of evidence indicating that the most subtle changes in questions can have a profound effect. Asking the same yes/no question twice, but reversing the emphasis, yields different results. For instance, people are more likely to answer yes to “Should smoking be banned in public?” than answer no to “Should smoking be allowed in public?”

Enter the Express. Its phone poll has two important characteristics. The first is that, for an Express reader, there is an obvious right answer. When asked “Are you fed up with fanatics changing Britain?”, or “Do you agree? Is Brown appalling?”, 99 per cent say “yes”. The second is that, just occasionally, the right answer is “no” (as with the ever-controversial: “Is it fair to make us work until we drop?”).

Here is where it gets interesting. If the research is correct, we would expect fewer people to answer “no” when no is the right answer than answer “yes” when yes is the right answer. And that seems to be the case. In a random selection of 40 previous polls, 32 are “yes” polls, and the average result is 98 per cent; in the other eight, the average result for “no” is just over 95 per cent. Performing a statistical check on the data, the Mann-Whitney U Test, it turns out that less than once in a hundred would such a difference happen purely by chance. People really do prefer to answer “yes”. Which means that the Express can stop those silly polls now.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s