London polling post-mortem

Most of the time on UK Polling Report I write about what the polls tell us about public opinion. Only when elections come along do we get to do it the other way around, and see what public opinion tells us about the accuracy of the polls.

As regular readers will know I have deep reservations about naive comparisons of eve-of-election polls and results and pollster "league tables". They are often used in an extremely simplicistic fashion, with people balancing absurd conclusions upon one pollster being one point closer in a poll with a three point margin of error. One can get a slightly better idea in a well polled election race when consistent trends can be identified (as Rob Ford, Will Jennings et al did at the last election, producing a very different league table) but ultimately all a pollster can really hope for is to be within the margin of error of the result. All else is luck.

Five companies produced polls for the London mayoral election (TNS-BMRB also did one poll, but it was early in the campaign so can't really be compared). Their final results are below.

YouGov, Opinium and ComRes were all within 3% of all the candidates' actual support, with YouGov slightly overestimating Brian Paddick's support, and Opinium and ComRes slightly underestimating Ken's. On the final round Opinium got the 52-48 split right, with YouGov calling 53-47 and ComRes 54-46. Well done to all three of them.

I'm unsure why Populus - who are normally one of the most accurate and reliable pollsters - ended up so out, showing a 12 point lead in the second round and significantly underestimating Livingstone in the first round. One thing that springs to mind is ethnicity. Since 2008 YouGov have weighted by ethnicity in London polls and it does makes a significant difference to results (Labour support ends up too low without it). There is no mention of ethnicity on Populus's tables... but then again, neither is there on ComRes's tabs. Perhaps Populus were just unlucky enough to get a dud sample. They also finished their fieldwork 4 days before the election, so perhaps there was movement towards Ken in those final days (the same applies even more to Survation, whose fieldwork ended 10 days before the election, so there was plenty of time for a swing).

Note that everyone overestimated UKIP's support... although part of that could be their decision not to put UKIP in their description on the ballot paper (though they still used the UKIP logo, which says UKIP in it.)

Survation and YouGov also produced figures for the London Assembly, figures below.