Dealing with likelihood to vote

Sunny Hundal asks a very sensible question over on Liberal Conspiracy having looked at the figures from MORI's latest political monitor and seen the massive difference filtering the poll by likelihood to vote makes.

While I certainly wouldn't dispute the importance of parties getting their votes out, things are a little different from how they appear, largely because of some of the differences in MORI's methodology compared to other companies. MORI take account of likelihood to vote in quite a strict way - they ask people how likely they are to vote on a scale of 1-10 and take only those who say they are 10/10 certain to vote. People who say they are 9/10 or 8/10 likely to vote, for example, are excluded. This means MORI's filtering by likelihood to vote has quite an extreme effect - as you can see from Sunny's post, this month the filter increased the Conservative level of support by 3 points and reduced the Labour level of support by 4 points.

However, other polling companies do it differently. ICM ask the same 1 to 10 likelihood to vote question, but weight by it rather than filtering. This means someone saying they are 9/10 likely to vote counts as 0.9 of a respondent, rather than being excluded entirely. This makes the effect of turnout much smaller - for example, in ICM's last poll turnout weighting increased the level of Conservative support by just 1 point, and made no difference to Labour.

Populus use a similar method to ICM. In their last poll, weighting by turnout had the effect of increasing Conservative support by 1 point and decreasing Labour by one point. Finally, YouGov don't weight or filter by turnout at all, so their published figures are the ones for everyone giving a view (during election campaigns YouGov does adopt turnout weighting, and typically the effect is similar to that ICM and Populus get - it increases the Conservative lead by 1 point).

So in conclusion, it only looks this way because Sunny's looking at MORI's polls in isolation - you won't find such a sharp contrast looking at any other company's results.

...

Those who have thought this through may be ahead of me now - in terms of the Conservative or Labour lead all the parties are reporting much the same results... but MORI's turnout adjustment makes a massive difference and no one elses does. Surely this means that, since their published results are much the same, their figures without turnout adjustment must be different? You'd be right - without a turnout adjustments or topline reallocations of don't knows, MORI's latest poll is showing a 10 point Labour lead, ICM's a 1 point Conservative lead, YouGov the two parties neck and neck and Populus a 3 point Labour lead.

The reason for this difference will be largely our old friend political weighting of samples - the majority of companies adopt some form of political weighting, normally recalled vote from the last election, to address what they see as an intrinsic bias towards Labour voters in phone samples. MORI do not, on account of their concern that levels of false recall may themselves change over time, making it unsuitable for weighting. The end result is that MORI's samples often contain a significantly higher proportion of people who said they voted Labour in 2010 than do ICM and Populus's samples - hence the difference in their voting intention figures when you take away the turnout adjustment.

For most of you, I'm sure this is largely academic - after all, the topline voting intention figures are much the same - but if you want to look under the bonnet of the figures like Sunny has, these things make a difference.