The Oldham and Saddleworth opinion polls

The full tables for the ICM and Populus Oldham East polls are now up here and here. As already dicussed, both have Labout 17 points ahead of the Liberal Democrats in second place.

Both use their standard methodologies, with all that it implies (past-vote weighting, weighting by likelihood to vote and re-allocation a proportion of don't knows to the party they voted for last time). The only significant differences between them are the sample size (Populus is three times bigger than ICM) and the voting intention question itself. Populus included the candidate names for the main three parties in the question, ICM didn't. There was some speculation prior to publication that this would make a significant difference to the result - it didn't, Populus and ICM are well within each other's margin of error.

We don't have detailled tables from Survation, the third company to produce a poll for Oldham East and Saddleworth, so we can't draw any firm conclusions about why their poll showed the Lib Dems doing much better. We can tell they had an exceptionally rate of people refusing to give a voting intention, and that they didn't weight by things like social class or tenure, which ICM and Populus did.

On the face of it, it looks as though Labour have it in the bag. I would add a few caveats to that. Firstly this poll itself has the potential to change the result. By demonstrating to supporters of the coalition that the Liberal Democrats the best placed to beat Labour, it could encourage Conservative tactical voting for the Liberal Democrats. Against that is that the Populus data suggests there is already a substantial amount of Conservative tactical voting for the Lib Dems, and there would need to be truly massive tactical voting to make up the large Labour lead.

Secondly, there were a substantial amount of former Conservatives and Liberal Democrats who said don't know or refused to give a voting intention. Overall, 24% of 2010 Tory voters did not provide a voting intention to Populus, a third of 2010 Lib Dem voters did not. ICM found a very similar pattern. While these voters have the potential to make a difference, remember they are already factored into ICM and Populus's topline figures, as they assume a proportion of them will vote for the party they did last time (in the case of Populus it reduced the Labour lead from 19 points to 17 points).

Finally I wanted to look a the record of by-election polls, which has the reputation of being somewhat patchy. Here is the record from the last Parliament:

In Norwich North there was a by-election on the 23rd July 2009 which the Conservatives won by about 20%. ICM conducted a poll in late June, so a month before the election, that was so early on it cannot fairly be compared to the result (it did, for the record, show the Tories winning).

In Glasgow North East there was a by-election on the 12th Nov 2009 which Labour won by almost 40%. There was a Scottish Opinion poll, but it was carried out in May 2009, so 6 months before the election (it did show an easy Labour win, but again cannot be compared).

In Glenrothes there was a by-election on the 6th November 2008 which Labour won by just under 20%. In September 2008 ICM conducted a poll showing the SNP and Labour neck and neck. Once again, it's too early to really be compared to the result.

In Glasgow East there was a by-election on the 24th July 2008 which the SNP gain the seat with a majority of 1.4%. There were two polls during the campaign. A Scottish Opinion poll on the 14th-17th July 2008 showed Labour ahead by 17%, an ICM poll on the 10th-11th July had Labour ahead by 14 points. Scottish opinion's fieldwork finished only a week before the by-election, so clearly got things wrong somewhere. ICM's fieldwork finished a fortnight before the by-election.

In Haltemprice and Howden there was just one poll by ICM, before the full wierd and wonderful list of candidates was known. It correctly showed David Davis walking it.

In Crewe and Nantwich there was a by-election on the 22nd May 2008, with the Conservatives winning by about 19%. It was heavily polled: there was am ICM poll a fortnight before the election showing a Tory lead of 4 points, then ICM and ComRes polls in the final week showing an 8 point and a 13 point Tory lead respectively. Notably the reallocation of don't knows to the parties they voted for previously, which works very well in general elections, did not help in this polls - it made them worse. Without them the ICM and ComRes polls would both have shown 15 point Tory leads.

Finally for the last Parliament, there was an NOP poll in Blaenau Gwent in May 2006, a month before the by-election, which showed Labour 12 points ahead. In the event Labour lost, but again, a month is such a long time in the context of a by-election the figures cannot reasonably be compared to the result.

So, while there were around 10 by-election polls in the last Parliament, the majority of them were conducted at the beginning or early on in the by-election campaign, so we can't really make any judgements about their accuracy. If they were wrong, it could easily be that public opinion shifted during the campaign (indeed, the same applies if they were right), although in some cases it appears that either the polls were wrong, or there were massive shifts in support between the poll and polling day.

The only cases where the polls were conducted close enough to the actual elections for us to hold them to account were really Crewe and Nantwich (where both ComRes and ICM correctly predicted a Tory victory, but underestimated the size of it) and perhaps Scottish Opinion's poll in Glasgow East (which showed Labour well ahead, when the SNP went on to win).