Why political polling is dying
Each week, YouGov conduct six political polls for News International. ComRes poll for the Independent about once a fortnight; ICM, MORI and Populus do monthly political polls; Opinium seem to be polling on a weekly basis; and Survation and TNS have irregular but frequent polls.
Compare this with 10 years ago. According to Mark Pack’s list of past political polls, this week in October 2002 had just two polls. Ten years before that, there was one poll in the same week. So political polling now appears in unprecedented health.
But perversely, political polling contains the seeds of its own destruction. Here’s why.
Polls are commissioned for one of two reasons. Either the commissioner wants to know something, or they want someone else to know something.
Most polls are never made public. They’re commissioned by companies that want to know how they’re viewed, or to test ideas, or to see what people think about a question that’s important to them. These are often of little interest to anyone who’s not directly involved, though sometimes they contain some fascinating insights.
However lots of polls are made public. Some of these are by organisations trying to create a story that helps their cause. A recent example was this Populus poll, commissioned by the Tories and credulously reported by the Guardian despite a question sequence carefully designed to give Ed Miliband a bad score.
But most public political polling isn’t done by campaign groups: it’s done by newspapers, and they do it because they want a good story that sells papers. This is where the problem is.
Suppose five different polling firms each interviewed five different random samples of 2000 people from the UK population on the same day, using exactly the same top-quality methodology. Due to natural effects of sampling we would expect the results to be different. They probably wouldn’t be wildly different but there’s a good chance that at least one of them would have a quite different result in the overall figures, and when you start digging into subgroups it’s almost certain there’ll be big differences.
This variation is excellent for newspapers. A result that shows change from a previous poll is a news story. Take ComRes’ poll for the Independent, published on the day of Miliband’s speech to Labour conference, and showing Labour’s lead as just 3pts. It made a fantastic headline: far better than they would have got if they had found the same result as almost any of the preceding seven polls by other firms, which was a Labour lead of between 9-13pts.
In short, newspapers are crossing their fingers that their exclusive poll will be the outlier, the statistical freak that screams “something’s changed”.
But the irony is, the more successful political polling has become as an industry, the less value there is for newspapers to commission polls. Yet newspapers are the group that keep political polling going*.
Back when polls came out just once a week, no-one could tell whether a poll that showed a dramatic change was a true reflection of opinion or just a rogue. Now, we can be pretty confident, within a couple of days at most, whether there’s been a change. It takes seconds, when hearing about a new poll, to go onto UK Polling Report and check how it compares with others. The emperor’s clothes are threadbare at best.
In case you’ve got to this point and think I’m seriously predicting the imminent death of political polling: I’m not.
Of course newspapers will carry on commissioning polls and hoping they’re lucky enough to get an outlier. Most of their readership only read one national paper and don’t go checking up on polling websites (more’s the pity).
But as readership trends change and people’s news comes from more individually-tailored sources, those things could change. People who have even a passing interest in the results of political polls will have increasing access to a range of polls, not just their own newspaper’s poll – if they even continue to have a newspaper they see as their own. The flaw in the model may yet be exposed.
—
* Polling firms make most of their money from non-political work and typically do the political stuff for the PR value. But if it wasn’t for the newspapers promoting their polls, it’s hard to see that they’d do political polling at all.
