Archive for August, 2014

This fracking poll finding is one of the least convincing I’ve ever seen

Posted in Bad polling, Energy sources on August 11th, 2014 by Leo – 5 Comments

A new poll has found over 3 times as many people support fracking as oppose it. That’s a reversal of previous polls, in which most people generally opposed fracking. So has there been a change in the public mood?

No.

Instead, Populus and UK Onshore Oil and Gas have published one of the most misleading poll findings I’ve ever seen.

Short of faking results or fiddling the weights or sample (which this poll doesn’t), there are two ways to get a poll to give the answers you want. You can ask a series of leading questions that get respondents thinking the way you want them to, then ask the question you’re really interested in. Or you can word the questions so respondents only see half the argument.

This poll does both.

The opening three questions are statements that form the basis of the argument for fracking. They’re phrased without any costs (free ponies for all), counter-arguments or alternatives:

  • The UK needs to invest more in a whole range of new infrastructure, including housing, roads and railways, airport capacity and new energy sources
  • The UK needs to use a range of energy sources to meet the country’s energy needs
  • Britain needs to be able to produce its own energy so it isn’t reliant on gas from other countries

Then comes the clincher. A question on fracking that’s 146 words long, describes the process with reassuring terms like “tiny fractures” and “approved non-hazardous chemicals”, and tells us that it could meet the UK’s natural gas demand for 50 years. No challenge to the data, no costs or consequences, no alternative energy sources.

This isn’t an attempt to find out what the public think about fracking. It’s message testing.

That’s what political candidates or businesses do before launching a campaign. They fire a load of messages at respondents to see how much support they could gain in a theoretical world where only their view is heard, and which arguments are most effective.

It’s a useful technique for finding out how people might respond to your arguments. But it’s not supposed to represent what people actually think now.

Except not only was this poll press released as if it shows what people currently think, it was reported as such by the BBC, Press Association and the Telegraph.

This is the kind of thing that destroys trust in polling. I can see why UKOOG wanted it, and I get that the journalists wanted a counter-intuitive story (though it’s a shame they didn’t question what they were given). But I’m surprised that a reputable pollster went for it.

TL;DR: