Is Euroscepticism collapsing, or is it just bad polling?

Today’s YouGov poll shows a startling change in attitudes to the EU. The results suggest more people would now vote to stay in the EU than to leave it: 40% staying in against 34% wanting to leave.

That’s a big swing from two months ago, when 49% said they would vote to leave: 17pts ahead of those wanting to stay:

Shifts like these don’t just happen by themselves. But is it real, or is something going on with the polling?

Option 1: a change in opinion

There are grounds for thinking a real shift has happened. The last time ‘vote to stay in’ was this high was December 2011: just after Cameron’s walkout of the EU summit.

At that time, the suggestion that the UK would leave the EU moved from remote to seeming more possible. Perhaps people started responding to the polling question differently: saying “I’d vote to leave the EU” became less of an empty threat.

Maybe that’s what happened this time as well. Over the last couple of weeks, discussions about the UK’s future in the EU have dominated the news again. People have started thinking about their own view, and they’ve responded to YouGov with a more considered opinion, which has taken some people away from the ‘out’ camp.

So we have a plausible explanation – but it’s not the only possible answer.

Option 2: bad polling

Some polling is designed to find out what people would do if they’re exposed to certain information or arguments. If Tesco promised to make its beefburgers with only British ingredients, would you be more likely to shop there? If you’re told that 60% of people affected by the benefit cap are in work, would you be more likely to oppose it?

But other polling is supposed to be a pure measure of what people currently think. Questions like voting intent and the EU referendum should be in this category.

So for the EU referendum question to show accurately what people think, respondents shouldn’t be shown anything that might influence their response. In an ideal world, they’d only be asked about the EU, and then the poll would finish. But that would be expensive, so we have to accept that the EU question will go in a poll with other questions.

In that case, the other questions respondents see need to be consistent between polls. So if respondents are being influenced by the other questions, at least it’s happening in a comparable way.

But that’s not how YouGov have done it.

The most recent poll, which shows ‘stay in the EU’ ahead by 6pts, asked six questions about Europe before it got to the referendum question. The previous recent high for Europhiles, in December ’11, asked 13 Europe questions before the in/out question. But the poll that showed ‘leave the EU’ the furthest ahead, in May ’12, didn’t ask any EU questions before it got to in/out.

Perhaps these other questions are affecting the result. The latest poll, for example, has several questions about renegotiating Britain’s terms of European membership – so respondents might be thinking of the in/out question in the context of revised terms. That could have prompted a few people to drift into the ‘stay’ camp.

Conclusion: has opinion changed?

There’s a plausible explanation for why opinion about an EU referendum might have changed over the last couple of weeks. As we apparently saw in December ’11, when the UK’s membership is in the news, people think about it more and become a bit less sure about wanting out.

But while this might make intuitive sense, there’s really no good polling evidence for it. The way YouGov have been asking the question has the potential to shift people’s responses in ways that are different from poll to poll. We can’t tell for sure whether it does skew things, but the inconsistent question structure means we equally can’t be certain that opinion really is changing.

I’d like to see YouGov move the in/out question to the start of their questions on the EU. Then we will be able to see what’s really going on with our views on the EU.


UPDATE 14.20 on 21.01: A commenter on Liberal Conspiracy (Roger Mexico) has pointed out that the previous week’s poll (Jan 10-11) asked the Europe question first (in fact, after whether or not respondents want a referendum), rather than after a long series of questions – and found rising support for staying in Europe.  This does indeed suggest that it’s not just a matter of “ask it first and people want out, ask it later and they’ll want to stay in”, and adds some weight to the argument that there is growing support for staying in.  It’s still not ideal to be moving the question’s position around so much, though!


UPDATE 9:15 on 22.01: YouGov have just published a new poll with the same question – this time asked before any other Europe questions.  It got a result of 40% leave vs 37% stay, so a swing of 4.5pts on the poll of four days earlier.  This would fit with the theory that the question sequence is affecting the result, but equally the previous poll (or this one) could be an outlier, which has nothing to do with the question order  (it’s possible that opinion really has changed this much in a couple of days but on such an established issue I doubt it).

  1. Dan says:

    You only have one years data but that also looks like Stay in goes up in the winter and down in the Summer,

    Probably just co-incidence but possibly Winter has chaos with a little bit of snow and they can handle it better on the continent stories.

    this summer we had feel good factor of Olympics.

    however more likely just coincidence.

  2. Gwen Tanner says:

    It doesn’t matter which way the questions are put down on paper. More information needs to be given out nationally the day BEFORE a poll with all current spending in the EU as well as the question “What if the other 26 European countries vote against giving the UK renegotiations on any directives or issues arising from them?”, I guarantee you would get an ‘out’ vote. Simple!

    • Leo says:

      Just on the basis of today, the terms as Cameron’s trying to position it seems to be “we’ll get a better deal so you can vote to stay; if we don’t, you can vote to leave”. YouGov’s latest poll (linked above) show how the response:

      “Imagine the British government under David Cameron renegotiated our relationship with Europe and said that Britain’s interests were now protected, and David Cameron recommended that Britain remain a member of the European Union on the new terms.”

      Got 53% vote to stay vs 26% vote to leave.

  1. […] Caught Out: YouGov changes question structure, then claims “opinion change” By admin On January 21, 2013 · Leave a Comment With apparent news that there has been a “shift” in public opinion concerning UK membership to the EU, evidence has come to light that Peter Kelner’s YouGov changed the question structure of their polls and then claimed an “opinion change“. Poll watcher Leo Barasi spotted it and wrote this: […]

  2. […] and then claimed an “opinion change“. Poll watcher Leo Barasi spotted it and wrote this: Indeed, analysis of the most recent poll, which shows ‘stay in the EU’ ahead by 6pts, […]

  3. […] polls and then claimed an “opinion change“. You can read Leo’s post and follow up on the Noise of the Crowd […]

  4. […] questions raised over YouGov’s reliability in the past. This was especially when they were  ”caught out” as they changed their question structure, then claimed an “opinion […]