European polling post-mortem

Time for a post-mortem of the European election polling. I'm not a fan of the sort of horse-race approach to these things - just because they are the final poll of the race, they still have normal margins of error, so if one pollster is a fraction more accurate than another it is often just the luck of the draw. Realistically the best a pollster can ever hope to do is get all the results within the margin of error. Better than that is luck. However away from the public "who won" stuff, comparing poll predictions to actual election results is an absolutely critical tool for pollsters - it's our chance to compare our figures with reality, to improve and finesse our methods.

The final polls from each company are here. Note that I haven't included Populus - they did conduct one European poll, but it was a fortnight before the election when just a week is a long time in politics! While I've included it in the comparison, one should allow some leeway for ICM for the same reason; their poll's fieldwork finished a week before the actual election.

The most obvious current difference between Westminster polls is the reported levels of UKIP - there is a big gulf between the levels of UKIP support report recorded by companies like ICM, MORI, YouGov and ComRes's phone polls and polls from newer companies like Opinium, Survation and ComRes's online polls. We don't know what the reasons for this are - there are a couple of things like prompting and re-allocating don't knows that we can account for, but mostly the difference is not easily explained. It may be something to do with interviewer effect, or the representativeness of different companies samples. We can't tell.

The European elections were obviously an opportunity to check figures against reality. I half expected the polls to all converge together in the run up to the election, as they have a tendency to do before general elections, but in reality we got the same sort of contrast as we do in Westminster polls. Higher figures for UKIP amongst newer online companies, lower figures from YouGov, lowest from ICM... and when the votes were counted the YouGov figure was the closest.

Of course, European elections aren't general elections. On the issue of prompting, for example, every company prompted for UKIP in their European polling, whereas only Survation do it for general elections. There were no telephone polls for the European election, so it can tell us nothing of them. European elections are low turnout elections, so some of the errors may have been down to too strict turnout filters (ComRes used a very strict turnout filter for Euros and would probably have been better if they'd used the method they use for general election polling. There was the issue of the Independence from Europe spoiler party on the ballot paper and so on. At a purely personal level though, getting UKIP right at the next election is the biggest challenge currently facing pollsters, so I'm relieved that in the first real proper national test we got it right. Phew!