MRP models and tactical voting

Given the success of the approach at the 2017 election I expect we'll see several MRP seat models this time round. The first one to emerge however is one constructed by Focaldata, using data from mixed sources, including YouGov, that Best for Britain have used to drive a tactical voting website. It has caused some controversy - particularly on the comment pages of the Guardian - with people arguing over the validity of its recommendations. I won't get too far into that (vote for whoever the hell you want), but thought it was probably worth making a few comments about MRP itself, considering it will crop up again through the campaign.

What is MRP?

First, we need to understand what MRP is. It stands for multilevel regression and post-stratification, which almost certainly doesn't help. There is a academic paper by Ben Lauderdale and his colleagues who run the YouGov MRP that explains it in great detail here, however the short version is that it's a modelling technique aimed at producing robust estimates for small geographic areas from large national samples. In the context of elections, that means coming up with estimates of vote share in single constituencies based on a big national sample.

Using traditional techniques even very, very large samples don't contain enough respondents to be a good guide to individual seats. If you had a huge sample of 50000, divided by 632 seats it would still give you less than 100 people a seat - which wouldn't be enough to produce decent data. I've seen this as a naive criticism of the Best for Britain MRP model (there are only 70 people per seat!) but in fact that is exactly the problem that MRP is intended to solve.

MRP works by modelling the relationship between demographic and political variables and voting intention (the multilevel regression part), and then applying that to the demographics and political circumstances in each individual constituency (the post-stratification). So in this case, an MRP model would look at how demographics like age, gender and education relate to vote intention, and how that differs based on political variables (Is there an incumbent MP? Is it a remain or leave area?). That model is then applied to the known characteristics each seat. What that means is the projection in an individual seat is not just based upon how respondents in that seat say they would vote, it's effectively also based on how respondents with the same demographics in seats with similar political circumstances say they would vote.

How well does it work?

At the last election YouGov had an MRP model that performed very well - correctly predicting the hung Parliament and some of the more unusual election results like Canterbury and Kensington. Clearly, given the accuracy of the YouGov model, it is possible to use MRP successfully to produce decent seat level estimates from a big national sample.

Best for Britain's defence of their tactical recommendations relies heavily on how well the YouGov MRP model did in 2017. However, not all MRP models are necessarily equal. It isn't one single model, it's a technique, and it's possible to do it well or badly. It is not certainly not a magical guarantee of accuracy. If we look back to 2017 the YouGov MRP model got all the attention, but it wasn't the only MRP model out there. Lord Ashcroft also commissioned an MRP model, but that wrongly predicted a Tory majority. Just as some polls have been more accurate than others in recent years, some MRPs may be more accurate than others.

The things that drive the quality of a MRP model should be the quality of the data that's going into it, and the quality of the model itself - have those designing it picked demographics and political factors that allow them to accurately model voting intentions? As an external observer however, it is quite hard to judge that. For the YouGov model there is its track record from 2017. From other MRP models, we're driving a bit blind. We know it is a technique that can be very successful if done well, but we won't really know if it is being done well until it's compared to actual election results.

Are tactical voting recommendations based on an MRP model sensible?

In principle, yes. MRP is obviously not perfect or infallible - nothing is - but it is an established technique for producing estimates of support in small geographical areas from a larger national poll. Certainly it should be better than using a crude uniform swing, or just basing recommendations on what the levels of support were at the previous election and assuming nothing has changed since.

In practice, of course, it depends on the quality of the model and the tactical decisions that people make based upon them - I certainly don't intend to get into that debate, especially since I expect there will be various rival tactical voting sites with different recommendation, and perhaps different aims and motivations.