UKPR Polling Average
Very soon I am going to launch a weighted average of the polls, the UKPR polling average. I've thought long and hard about this because generally speaking I don't like polling averages. There is no statistical justification for a polling average - the different companies do slightly different things, they weight differently, ask different questions and include people who are more or less likely to vote and more or less certain for whom. Averaging them together isn't the equivalent of one big poll with a smaller margin of error, it's just mishmash of different methodologies. Neither does an average get you the better results - the true picture isn't normally the average of the polls, in fact, when compared to elections the poll that's worst for Labour tends to be the best.
So, with all that in mind why am I doing it? Two reasons: the first is that there is demand for it, and if I don't provide it other people will, and will do it less well. My firm belief is that the best way to follow the polls is to look at individual pollsters, see what their trends are and understand the differences in approach that result in the differences between them. I hope that regular readers here will always judge polls in this manner. The reality is that, especially as we approach the next election, is that many people (and newspapers!) can't be bothered to do that, and just want one nice figure that shows them what "the polls" show. Since someone will provide it, I thought I'd better do it properly.
A second reason is that it allows me to put up a running projection of what the current polls would translate into on a uniform swing. At the end of the day, the one question a lot of people want to know the answer too when they look for polls is "if there was an election now, what would the result be?" To do that, we need a figure representing what "the polls" show.
In the past there have been a couple of different approaches to polling averages. The first is a straight rolling average of the last five polls - what the graph on my voting intention page shows. That is vulnerable, however, to being skewed by having a lot of polls from a particular pollster in it. Secondly you can take the average of the latest poll from each company. That has the downside of what you do with companies who poll irregularly, or what happens if they miss a poll. You also miss out on good quality data from pollsters who produce lots of polls, while include aging stuff from pollsters who produce data less regularly (as Nate Silver, who produces the US averages on www.fivethirtyeight.com said when he justified including data from more than one poll from the same pollster in his averages "getting SurveyUSA's sloppy seconds may be as good as getting virgin results from a lot of pollsters"). Another quite common approach is to weight data according to sample size, something I particularly dislike since in the past the polls with the largest sample sizes have certainly not been the most accurate (ICM, for example, who have one of the most enviable records, normally have the smallest sample size).
What the UKPR Polling average will do is weight the polls that go in according to how recent they are and the track record of the polling company. They will also factor in some methodological and transparency issues and whether there are other polls of the same company in the average (as a compromise to stop mulitiple polls from the same company having too much effect). I'm never going to like polling averages, but given they are going to exist, I can at least provide the very best one I can.
