More on that mruk poll

On Sunday there was a mruk Cello poll on the London elections and we needed to be a bit wary as we had neither track record nor methodological details to judge it by. Well, their full tables are now up on their website here so we can have a proper grub around in their methodology.

The poll was a phone poll. It was weighted by normal demographics, ethnic group (as all the London polls in the campaign have been apart from YouGov's early ones) and recalled 2005 vote. The past vote weights are in practice almost identical to ICM's in terms of party support (actually they should in theory be marginally kinder to the Tories, but the difference is a tiny fraction of a fraction of a percentage point, so can be disregarded).

That's not to say the samples are identical, there is actually a vast difference in terms of the proportion of non-voters in the sample. Comparing the samples between mruk and ICM, mruk's is weighted to contain far, far more people who claim they voted in 2005. In ICM's weighted sample 44% said they did not vote and 4% refused to say or didn't know. In mruk's sample 21% said they didn't vote and 7% refused or wouldn't say, so only half as many 2005 non-voters as ICM.

This probably isn't all down to weighting, but down to how the question is asked. In the two companies raw, unweighted data 33% told ICM they didn't vote or weren't registered but only 20% said they same to mruk - my guess is that ICM used wording that prompted people to make it more socially acceptable to admit that you didn't bother to vote in 2005. Either way, the impact on final voting intention figures probably isn't huge, people who didn't vote in the general election are probably least likely to vote in the mayoral election.

Having weighted the data, mruk applied a turnout filter that took all respondents who said they were 8/10 likely to vote. This is slightly more demanding than ICM's normaly filter, but laxer than Ipsos MORI's. This equated to 73% of the sample in this case.

I suspect it is identifying which people are likely to actually go out an vote that is the main challenge in elections like this with a lower turnout. Just asking people to rate their chances from 1 to 10 has limitations since even the proportion of people who say they are 10/10 absolutely certain to vote is often higher than the proportion of people who actually vote. In ICM's case they pre-empt their question with wording intended to coax people into admitting that they might not vote: "Many people we have spoken to have said they will NOT vote while others have said they WILL vote. Can you tell me how certain it is that you will vote?" and found a much lower proportion of people claiming they were 10/10 certain to vote. (Likelihood to vote doesn't seem to work the same way in YouGov polls - in polls on low turnout elections their figures without any filtering by turnout have in past been far more accurate than ones with)

The bottom line for those wondering whether this is a reputable poll that we should pay attention to is yes, there's nothing wrong with the methodology. That said, if the polls continue to produce contrasting figures then (unless the result in bang in the middle and everyone can claim they were within the margin of error)

someone is going to have been wrong come May 2nd.

I'm told that there will be another mruk poll in the Sunday Times this week.