Why do Angus Reid polls show a lower level of Labour support?
Every fortnight or so there is an Angus Reid poll for PoliticalBetting, and every fortnight or so there is the same discussion in the comments here about whether they are right or wrong. Why do Angus Reid consistently show a lower level of support for Labour?
There probably isn't one simply answer, I've seen several people spot a potential difference in what Angus Reid do, and decide on that basis that this *must* be the cause of the difference. Realistically, there are a lot of differences between what different pollsters do, and all these things can have an impact on the result. So -
Angus Reid word their question differently (from all the other companies) Angus Reid do not have voting intention as their first question (unlike all the other companies) Angus Reid do fieldwork online (unlike ICM, MORI, CR and Populus) Angus Reid use their SpringBoardUK panel (unlike YouGov) Angus Reid weight using newspaper readership (unlike ICM, MORI, CR and Populus) Angus Reid weight using past vote (unlike MORI and YouGov) Angus Reid weight using data collected at the time of the survey (unlike YouGov) Angus Reid do not factor false recall into their past vote target (unlike ICM, Populus and CR) Angus Reid use a squeeze question (unlike ICM, Populus and YouGov) Angus Reid do not filter or weight on likelihood to vote (unlike ICM, MORI, CR or Populus) Angus Reid do not reallocate Don't knows (unlike ICM and Populus)
This is not to suggest that Angus Reid are more different than other companies - you could come up with lists of exactly the same length for any of the other polling companies. It's just to demonstrate how variable the methodologies used by pollsters in voting intention questions are. For an outside observer it can be very difficult to isolate a variable and see what it's effect is, especially since many of the differences are inter-related and have effects that cancel one another out.
Anyway, with those caveats aside, what are the most likely explanations for the difference? We can largely discount the effect of the question wording - my understanding is that Angus Reid have done parallel testing of the different question wording and it makes no difference to the result. We can also ignore the lack of a filter/weighting on likelihood to vote, since the experience of all other pollsters is that introducing this would make Angus Reid's Labour figure even lower. We can also dismiss the squeeze question - Angus Reid's tables show their results prior to the squeeze question being asked, and it does not normally make any difference to the level of Labour support.
Angus Reid's decision not to ask voting intention as the very first question has the potential to skew responses (which is the reason no one else does it!), but I would be surprised if it had much effect in this case - most important issue is a non-partisan question that should not skew answers.
One factor that almost certainly makes a difference is Angus Reid's weighting. The other three companies who weight by past vote target figures that assume a certain level of false recall (in practice, this means Labour are weighting to a higher level and the Lib Dems are weighted lower). In contrast Angus Reid weight their past vote to figures based on the actual shares of the vote in 2005, meaning their samples have a higher proportion of people who said they voted Lib Dem and a lower proportion of people who said they voted Labour than ICM, Populus or ComRes. They also weight people to who say they did not vote in 2005 to a lower level - this too is probably a factor, since the Conservative lead amongst those people seems to be lower.
Angus Reid's tables do not provide a cross break by past vote, so we can't work out for certain the difference it makes, but we can get a good idea of the potential impact. If you look at the last ICM poll and take their answers before likelihood to vote filtering and their topline adjustment, it gave shares of the vote of CON 39%, LAB 31%, LDEM 20%. Now, if you crudely reweight the past vote to the figures Angus Reid use you get CON 41%, LAB 30%, LDEM 21%. As we'd expect, Angus Reid's weighting helps the Lib Dems and Conservatives and punishes Labour.
However, this doesn't explain the whole difference, since Angus Reid do not in fact show higher levels of Conservative support , but they do show higher levels of support for "others" - the minor parties like the BNP, UKIP and Greens. What I suspect is that there are two factors at play here - *something* is pushing up their level of other support, and the remaining support is being skewed against Labour and towards the Conservatives by their past vote weighting.
As to what that *something* causing a higher level of support for others is, I really don't know. I can't see any obvious cause. It could even come down to Angus Reid's sample, or the make up their panel - things it's not really possible to judge from outside.
This brings us to the question of who is likely to be right. Several people on here have said that you can't judge at all until an election, others have said that a pollster who is consistently different from all the others is more likely to be wrong. I wouldn't necessarily agree with either. In 1997 ICM were consistently different from other companies and turned out to be correct. In principle it can be the outlying company that's right and everyone else who is wrong, but an important caveat is that in 1997 we knew from the previous election that there was a problem, ICM turned out to have solved it and others hadn't. In 2005 all the pollsters did pretty well, so for Angus Reid to be correct everyone else (despite their differing methods) would have had to have developed some problem with their polling since 2005 (or be unable to cope with changes in public opinion since then...)
The other question was whether you can judge a pollster before an election. In some cases you can. If, for example, a new pollster came along who weighted their sample so it was 75% male and 25% female we wouldn't need to wait for an election to know they were doing something pretty seriously wrong. The failure of Angus Reid's past vote weighting to account for false recall seems demonstratably wrong in the same way - the evidence of false recall is solid. Since Himmelweit et el first described false recall of past vote based on panel studies in the 1960s and 1970s it has been repeatedly tested in academic panel studies and we can be relatively certain that there a systemic skew in how people report they voted at previous elections. Weighting recalled past vote to the actual shares of the result should underestimate Labour support.
My view therefore is that Angus Reid are likely to be wrong in terms of the gap between Labour and the Conservatives. However - other polling companies have tended to overestimate Labour's position, so we may yet find that Angus Reid are right for the wrong reasons. It's also worth noting that the academic evidence for false recall is largely based upon face-to-face surveys, theoretically people could be more accurate in their recall in online polls. In terms of the other big difference between Angus Reid and the other pollsters, their higher level of support for "others", I have little guidance I can offer you - I don't know what the cause of the difference is, so can't begin to say if they are right or not. Only time will tell.
Of course, this is just my opinion - I'm sure Angus Reid would disagree. No pollster produces figures that they themselves think are wrong, especially on voting intention, figures upon which reputations are built and destroyed. I normally try my best to explain the potential differences between polls rather than say which one is wrong or right - after all, I work for YouGov, so I have a natural bias in that direction. I think one can come to a legitimate view on which poll will be right ahead of an election based upon their methodology... but even then, you won't know if that view is right until the day afterwards.
(BTW - there will indeed be a brand new ComRes poll for the Indy on Sunday later on tonight)
