BBC report findings from a sub-sample of NINE

Years ago I slated the BBC for commissioning a poll about religion and trying to draw conclusions about people from different religious groups using pathetically small sample sizes. They are at it again in this poll.

The ComRes poll found that 63% of people agreed with the statement that "Our laws should respect and be influenced by UK religious values", which is fair enough. The BBC report then goes onto say that "A significantly greater proportion of the Muslims and Hindus polled (albeit in relatively small numbers) supported a strong role in public life for the UK's (essentially Christian) traditional religious values."

If we look at the tables though these weren't "relatively small numbers", they were minute numbers. The poll interveiwed 21 Muslims and 9 Hindus. The "significantly greater" figures weren't significantly different at all - 79% of Muslims agreed and 74% of Hindus agreed. The margins of error on these tiny groups are something like 23% and 32%, so they are not significantly different at all.

So, to the BBC, if you want to compare the views of different religious communities, you need to commission a bespoke poll taking representative samples from each religious community you are interested in. You can't do it for a cutdown price on an omnibus and try to say something about British Hindus based on nine interviews. If you just want to take the views of Britain as a whole, don't try to draw conclusions about tiny subsamples that your data cannot possibly support.

I suppose we should at least be grateful for that small caveat about sample size and that the tables are there on the BBC site to see, since the BBC story then gets picked up by other people who report it even more badly. For example, the Telegraph make the findings about Muslims - based on 21 people remember - the headline of the story. The Daily Mail surpass that by also highlighting that "three-quarters of Sikhs said..." There were 3, that's THREE, Sikhs in the sample.