The Greens Aren't on 13% - Here's One Reason the Polling Might be Wrong

Polling Twitter has been awash with discussion of one poll, from PeoplePolling, which put the Green party on 13%. Based on the weight of evidence, we can be confident this finding isn't accurate. Polling averages, ours included, put the party on less than half the figure, other pollsters' estimates range from 2-7% and there has been no evidence of a resurgence in national by-election results. There is also little reason to suppose the Greens might be seeing such a groundswell in support from political developments.

What's more, it's difficult to attribute this divergence to mere error or noise. PeoplePolling report their own margin for error at 4% and such a large divergence from the mean, of over 100%, is not attributable to sampling inconsistencies. Further to this, PeoplePolling have been consistent in overstating the vote share of the greens, relative to other pollsters. The next largest estimate for the Greens, 10%, comes from the same pollster a week earlier. So, what explains their figure?

One explanation could be their sampling methodology. As pointed out by Anthony Wells, the description of their sample source as a "a panel provider offering participants the chance to win money" certainly suggests the source is FindOutNow. Based on this assumption, it's possible to speculate as to at least one potential source of error.

FindOutNow has previously published a report on their election polling which did suggest that, even when weighted by standard measures, their figures overstated the vote share of minor parties and somewhat understated that of the Conservatives. Although this was not of the same margin, it seems similar to the direction divergence of PeoplePolling from the average. FindOutNow overstating the vote share of smaller parties, particularly the SNP, is something I have previously covered in the context of polling on Scottish independence. It seems plausible their sampling methodology could account for some of this divergence.

FindOutNow relies on panel data from Pick My Postcode, a site open to all which gives access to free giveaways. Although not a problem isolated to them, the panel is somewhat unrepresentative - favouring women and older voters.

FindOutNow do, of course, weight their samples to make them nationally representative. However, it's possible that their weighting measures are failing to capture some source of sampling bias. From the direction of skew, favouring smaller parties, it seems plausible that samples are more politically engaged. Though, again, this is mere speculation and by no means unique to FindOutNow.

Taking a look into polls conducted explicitly based on FindOutNow data, and we do see some of the same patterns - particularly, as previously mentioned, in Scotland. However, other national samples, including those conducted alongside Electoral Calculus, don't exhibit the same magnitude of divergence as PeoplePolling - suggesting there is something unique to their specific methodology, perhaps as related to FindOutNow.

It's difficult to probe more closely into the sampling practises of some other pollsters as, unlike weighting data, there is far less publicly available transparency data. Either way, it only adds to the imperative of considering polls within the context of the broader field.