Lib Dem private polls again

Back to those Lib Dem private constituency polls they were so keen to brief the press on last month. When we left the story it had emerged that the polls had been done as purely field & tab, so there was no British Polling Council requirement to release them (that is, Survation just made the phone calls, the actual method and analysis was done by the Lib Dems themselves. In the same way, for example, ICM or Populus's call centres are sometimes commissioned to do the phone calls for other companies and it doesn't make it an ICM or Populus poll). Now the Lib Dems themselves have released one of the polls, in Hornsey and Wood Green - tables are here.

Hornsey and Wood Green is the seat of Lynne Featherstone and has Labour in second place. Lord Ashcroft's polling of Lib Dem seats shows them doing far worse in LD -v- Lab seats than in seats where they are up against the Tories. In Hornsey and Wood Green itself Ashcroft found the Lib Dems trailing by 13 points. The Lib Dems own poll shows them only behind by one point.

As ever, one should be particularly suspicious of polling produced by political parties - certainly one should ignore them unless they cough up the actual tables so we can see what they've been up to. In this case the Lib Dems have done just that, so we can see for ourselves. As we knew from Survation's earlier statement, the polls were not conducted using Survation's own methodology, but using a methodology directed by the Liberal Democrats. There were a couple of specific factors:

• The first was that the poll was weighted to actual past vote. There are essentially three approaches to past vote weighting in constituency polling. One is to assume the problems of false recall and people moving constituency are insurmountable, and not to use past vote weighting at all. This is what Survation do in their own polls. Another is to estimate some level of false recall and factor that into your weighting targets, which I think is what Ashcroft does in his constituency polls. The third approach is to assume no false recall and weight to the actual shares of the vote in 2010, which is what the Lib Dems did.
• The second is that the question itself included the names of the party candidates, which is generally thought to boost the Lib Dems. This is not necessarily a bad thing - certainly I have grave doubts about polls done in Lib Dem constituencies that just ask a standard voting intention question. The alternative approach taken by Lord Ashcroft is to ask a two stage question, asking people a standard voting intention question and then to think about their specific constituency. This also gives the Lib Dems a significant boost.
• The third was question order. The Lib Dems did not ask voting intention first, instead they asked people to rate Lynne Featherstone first. It is possible that this boosted her personal vote in the subsequent voting intention question.

All three of these methodological choices probably helped the Lib Dems. However that doesn't necessarily mean they are wrong. I wouldn't personally have made the same weighting decisions as them, or asked a question before the voting intention question, but that's the beauty of publishing the tables for people to see: we can judge for ourselves what to believe. The candidate naming question is perfectly reasonable - the fact is, because constituency polls have historically been very rare we don't really know what the right approach is. It may be that a raw standard question actually gets the best result, it may be that the two-stage question with a constituency prompt gets the best result, it may be that prompting with candidate names gets the best result, it could even be that prompting people to rate their MP first gets the best result (though my own guess is that it is forcing respondents too strongly to consider the candidate). Unless someone is nice enough to commission and release lots of constituency polls using different approaches very close to the election, we will never know.

With all that said, I'll still be giving no real weight to the Lib Dem private polls for completely non-methodological reasons. I would advise people to ignore them because of potential publication bias. The Lib Dems have commissioned about 120 of these polls, they've released one. I don't think it overly cynical to ponder whether they may have chosen to release one putting them in a relatively good position, rather than one showing them getting a soaking. With sample sizes of 400 or so people, the Lib Dem polls have a margin of error of +/-5% (and that's on each figure, so on the lead between one party and another it's double that). By definition, normal random sample error means some of those polls will be overstating the true Lib Dem position by as much as 5 points, others understating it by that much. We have no way of knowing whether there are many other private Lib Dem polls that show the party doing even worse than in the publicly available polls, which they have chosen not to publish. And, to be fair, short of publishing all their private polling there is no reasonable way the Lib Dems can prove it either.