Polls tonight - and a word about wording

Two new polls tonight - there is a new ComRes poll in the Indy on Sunday, and the weekly YouGov poll for the Sunday Times.

Mike Smithson has a post up criticising the wording of the questions in the ComRes/Indy on Sunday poll. ComRes's polls for the Indy group almost always use the same format - do you agree or disagree with a set of statements, using statements that are often worded in quite a partisan way. It's not the first time they've come into criticism for it.

Now, I deliberately try to avoid writing posts here criticising the wording of particular questions in polls, or at least, not unless they are utterly shocking. The main reason is that no poll can be perfect and pollsters try their level best to word questions in as fair and even way as possible. There will be lots of questions that I'd have worded slightly differently if I'd have been writing them - but I'm sure other pollsters would write some of my questions slightly differently too (hell, sometimes in hindsight I'd write some differently). More to the point, bias is in the eye of the beholder - almost always people who don't like a poll can convince themselves that the results don't count because the wording is biased, and there's no much you can say to convince people otherwise. I have no desire to get into arguments over whether this or that question could have been worded slightly differently.

So I'm not going to comment on the particular questions on the ComRes poll, but I did want to address the style of questions they use and the purpose of doing it that way. I often write questions like that myself - that is, a bank of statements each from a clear political direction, asking people to agree or disagree with each one. The purpose of it is to see how many people sympathise with political arguments that might be put forward, arguments that may be comparatively complex or use partisan rhetoric.

Often if you want to test out the sort of partisan arguments being used by the parties to see which the public sympathise with, and this is a good way of doing it. Another good way is to give people two opposed statements and ask which best reflects their view, or - similarly - word a question that ascribes these views to "some people" - i.e. something along the lines of "Some people have said that X, while others say Y, which do you agree with".

To give a recent example, asking questions about whether people perceive the Lib Dems as betraying their principles by entering coalition. Asking a straight question of do you think the Lib Dems have betrayed their principles would sound a bit biased - there's no nice neutral way of wording "betraying your principles". Hence the best way is probably to ask if people agree or diagree with a statement of "The Liberal Democrats have betrayed their principles by going into coalition with the Conservatives", and stick it in a block of questions along with some that are more positive, such as "The Liberal Democrats did the responsible thing by agreeing to coalition at a time of crisis" (and indeed, that's how I did do it.) Therefore you've tested both the argument the Lib Dem's critics would use, and some Lib Dems defences, and you should get a good overall picture.

That said while this question structure has its purposes, it isn't the best choice for every question (personally I tend to use it pretty sparingly) and I wouldn't necessarily use it for lots of the things the Indy asks about.

Anyway, I'll post later when results are out - ComRes is due at 7.30pm, YouGov at 10pm.