Category Archives: opinion polls

Fisking Nate Silver

Nate Silver Gesturing Wildly. Actually, Just a Slow Shutter Speed.
Nate Silver Gesturing Wildly. Actually, Just a Slow Shutter Speed., a photo by handcoding on Flickr.
The Scotsman today published an interesting interview with the American psephologist Nate Silver. Let’s have a look at the interview part of the article:

Nate Silver, the award-winning statistician who shot to fame when he correctly predicted the outcome of all 50 states in the 2012 US presidential elections, says all the indicators point towards Scots voting to stay in the UK on 18 September next year.

True, although some recent polls have been very close (37% Yes vs. 46% No in the Panelbase poll for the Sunday Times in July, and 34% Yes vs. 36% No in the indirect question asked by Panelbase for Wings over Scotland), and although the gap has been closing slowly.

Only a “major crisis” south of the Border could turn the situation in favour of independence, despite it being more than a year until polling day, he added.

If the Panelbase polls are correct, only 0.5% of Scots need to shift from No to Yes every month for the Yes side to win. That’s clearly possible without any major crisis (we’ve seen much bigger shifts than that in electoral campaigns lasting less than a month).

In an interview with The Scotsman, Mr Silver said polling data was “pretty definitive”. “There’s virtually no chance that the Yes side will win”, he said. “If you look at the polls, it’s pretty definitive really where the No side is at 60-55 per cent and Yes side is about 40 or so.

Now this is odd. To put Yes at 40% and No at 60% means he’s excluding the Don’t Knows — which is fine — but if you look at the most recent polls in that way, we get this:

Poll Yes No
Panelbase, WoS, August (indirect) 49% 51%
Panelbase, Sunday Times, July 45% 55%
Panelbase, Sunday Times, May 45% 55%
Ipsos MORI, Times, May 34% 66%
TNS BMRB, April 37% 63%
Panelbase, Sunday Times, March 44% 56%

I don’t know about you, but it looks to me like Nate Silver is for some reason completely ignoring the Panelbase polls in order to reach his conclusion here.

“Historically, in any Yes or No vote in a referendum, it’s actually the No side that tends to grow over time, people tend not to default to changing the status quo.

Really? Let me quote from the Wikipedia article about Quebec’s 1995 referendum: Early polls indicated that 67% of Quebecers would vote “No”, and for the first few weeks, the sovereigntist campaign led by Parizeau made little headway. […] Under Bouchard, the numbers continued to change; new polls eventually showed a majority of Quebecers intending to vote “Yes”. […] Days before the referendum, it appeared as if the sovereigntists would win.

As noted on SCOT goes POP!, there are several examples of independence referendum campaigns developing like this. He offers this explanation: My guess is that, paradoxically, the more important a referendum is, the less likely voters are to swing to No by default. The supposed tendency that Silver talks about is largely a side-effect of electorates so often being faced with relatively trivial matters in plebiscites.

“The No side is even more dominant with the younger voters, so there’s not going to be any generational thing going on.”

Really? Panelbase’s July poll had 48% Yes vs. 52% No for 18–34 year olds (after excluding the Don’t Knows), 50% Yes vs. 50% No for 35–54 year olds, and 36% Yes vs. 64% No for 55– year olds.

Most polls I’ve seen have confirmed this: The youngest voters are slightly more sceptical than those in their 30s and 40s, but the oldest voters are the ones most likely to vote No.

Speaking yesterday, Mr Silver said, however, that the Yes campaign could benefit if there is some kind of dramatic economic collapse south of the Border.

“If there was a major crisis in England – if the Eurozone split apart and there were ramifications economically (for the UK) – the maybe things would reconsidered a little bit.

Perhaps. There’s also a strand of thought that people are more likely to vote for independence if they feel positive and optimistic about the future.

But he added: “For the most part it looks like it’s a question of how much the No side will win by, not what the outcome might be.”

This is a rather bold conclusion, given the figures I quoted above.

The French-speaking province of Quebec in Canada has previously rejected a vote on independence, despite sharp “cultural differences and genuine hostility” with the wider Canadian state, Mr Silver added.

I’m not sure what he’s trying to conclude here. Surely he doesn’t believe you can only win an independence referendum if there’s sufficient “cultural differences and genuine hostility”?

“That is a case where a smaller country reads more about the economic consequences and it becomes harder to change the status quo.

As far as I’m aware, the Quebec referendums failed because they didn’t manage to convince the Anglophones they wouldn’t be discriminated against in a French-speaking country, not because they read about the economic consequences.

“That was one where the Yes vote had been ahead, then faded down the stretch and lost.

The Yes vote was well ahead until two weeks before polling day, and in the end the result was 50.58% No vs. 49.42% “Yes”. I don’t think that’s called “fading down the stretch”, it’s called “losing the final sprint”.

“So on general principle, even if you took all the undecided votes, they are more likely to end up being No votes than Yes votes.”

“On general principle”? Where did that come from? My experience from canvassing tells me that most of the undecided voters would like to vote Yes but are looking for reassurance. As a young man told me on the doorstep, after placing himself at 5 on a scale where 1 is No and 10 is Yes: “An independent Scotland could be so good!”

The flatness of the polls

Opinion polls compared.
Opinion polls compared.
The Panelbase poll that was published today was basically another independence referendum opinion poll that didn’t register any statistically valid voter movements (given the sample size, plus or minus one or two points is definitely within the margin of error).

The dramatic changes we sometimes hear about come about because people compare the results from two different pollsters — if you look at one at a time, the polls have been practically static for months.

But why is that? My beloved wife offered the explanation that most voters actually aren’t very interested (yet), so they actually don’t pay much attention to all those fascinating stories that all the activists assume must be discussed at length over Scotland’s dining tables, which is why they don’t change their opinion, no matter how shallow it is.

If this is true, we all need to get better at talking to those uninterested voters and make them understand why this is so important, so that we can get the polls to start moving again!

Brace yourself for the next Ipsos MORI poll!

Opinion polls by pollster.
Opinion polls by pollster.
Ten days ago when Ipsos MORI had Yes on 31% and No on 59%, lots of Yes campaigners were a bit disheartened, while Better Together were celebrating. Today the roles are reversed because Panelbase have published a poll where the split is 36% vs. 44%.

What’s going on here? Surely 15% of the Scottish public didn’t get convinced by the Yes campaign in just ten days?!?

To find out, I plotted the recent opinion polls, divided by pollster (see the graph above). Ipsos MORI’s polls are displayed in blue, and Panelbase’s are in red.

It’s clear that Ipsos MORI are consistently finding many more No voters than any other polling company. Interestingly, they find the same amount of Yes voters as everybody else, so they must somehow get more undecided voters to come out as No voters than the the other pollsters.

Panelbase are also clearly finding more Yes voters than anybody else.

It’s worth pointing out at this point that we don’t know who’s right. We can’t assume the average is correct — for all we know, even Panelbase might be underestimating the number of Yes voters. Because there has never been a referendum on Scottish independence before, we just have no empirical way to rate the different methodologies employed by the polling companies. After the actual referendum has taken place, we’ll be able to rate the different polls, but at the moment it’s a bit of guesswork.

Anyway, the real point here is that we shouldn’t compare the last opinion poll (from Panelbase) with the one before that (from Ipsos MORI), because the systematic difference between the pollsters is far greater than the actual poll movements.

It’s a much better idea to look at each polling company separately. For instance, the last three Panelbase polls had Yes on 34% — 36% — 36%, and No on 47% — 46% — 44%, so there’s been a slow movement from No towards Yes.

So when Ipsos MORI publish their next poll, we’ll have to compare it against their previous one, not against Panelbase’s very different figures.

Yes will overtake No in five months’ time

The independence polls
Independence opinion polls with trend lines.
OpenOffice is able to compute trend lines for data in a graph.

I took the most recent Scottish independence opinion polls (since August last year), put in the trend lines and extended them until the day of the actual referendum, 18th September 2014. The results can be seen in the graph on the right.

Basically, No is falling quickly, while Yes and Don’t Know are rising at a more moderate pace.

The effect is that according to current trends, Yes will overtake No on the 1st of September 2013, and by the time of the referendum, there will be more than twice as many Yes voters as No voters.

Obviously it’s very unlikely that the current trends will continue for the next 18 months, and of course opinion never shifts in such a neat, linear fashion. However, it does show that the Yes side is building up momentum, and that it is indeed possible, perhaps even likely, that Scotland will vote yes to independence.