Regulation of Political Opinion Polling Bill [HL]

Lord Lipsey Excerpts

Friday 19th June 2015 Lords Chamber

[Lord Lipsey (Lab)]

My Lords, I am one of those whom the noble Lord identified who now claim to have got something right during the general election when the polls were getting it wrong, but in my case I can document the fact. I did not get the result right—I had a substantial bet on a Labour-minority Government, though I did get it at 5:1 before it went back down to 11:8—but I recovered my money from a small bet on the Liberal Democrats getting fewer than 10 seats. Sometimes your money goes where your heart lies.

What I did during the campaign, maddened by the overinterpretation of and reliance on the polls of the Guardian, was to write to point out in moderate terms—and wearing my hat as chair of the All-Party Group on Statistics—the limitations on polling, which I shall come to. So I can document it but not through showing people the pages of the Guardian, because it declined to publish the letter. I was not totally surprised because it was paying out a lot of money to commission polls and then oversold them like mad, so of course theGuardian was not very keen to publish this letter—although senior members of editorial staff say, in retrospect, that they think they should have done. Anyway, that is quite long enough on “I was right all the time”.

The performance of opinion polls in recent elections is worse than has been appreciated. Over the last six general elections, the opinion polls have got it completely wrong twice, a 33% rate. If they used that other invaluable tool of forecasting, a pin, they would have got a 50% hit rate. The amount by which you gain in having the polls over the use of a pin is not very great, but it is worse than that. First, although they got the winner right in four cases out of six they did not get the share of the vote for each party right in those cases. For example, in 2010 they overestimated the Lib Dems’ share. Secondly, even when they got right, it was often the result of pure chance. I went to a fascinating academic conference on the 2010 election, which, to summarise briefly, showed that the polls made a series of mistakes which happened to cancel each other out.

In the light of all that, I welcome the Sturgis inquiry, whose session the noble Lord, Lord Cooper, and I are both attending this afternoon. In preparation for that, I reminded myself of the study by the Market Research Society into the failure in 1992. Funnily enough, I am not sure that this inquiry will come out with anything very different. It identified three main sources of error: first, shy Tories, or people who did not like to say that they were voting for the Tory party; secondly, failures in the quotas, which I will come back to, which meant that they were not using representative samples; and, thirdly, differential turnout. Those were the main causes. Professor Sturgis may not get much further forward, although I hope he will.

The real question underlying this is whether polls technically can achieve the degree of accuracy that is necessary when they forecast elections. We had quite a lot of polls during the election, for example, that said something like Labour 30% and the Tories 30%. That could mean, within the statistical margin of error, Tory 33% and Labour 27% or the other way round. In other words, it could mean a 6% Labour lead, a 6% Tory lead or that they genuinely were neck and neck—and that is only in 19 cases out of 20, with the 20th case being even further out, as I think was true of the famous YouGov poll in the Scottish referendum.

That is the statistical margin of error, but it is only the beginning of the margin of error, as it assumes that you have a perfect sample which is representative of the electorate as a whole. But you do not have a perfect sample; you have a less and less perfect sample, as the noble Lord, Lord Foulkes, has pointed out. If you do it by telephone, you have the increasing problem of mobile telephone ownership, which can distort the figures. You can do it on the internet, but not everyone has access to the internet. You can also have what is called panel effect, whereby because you are asking the same people they become more informed and more interested, and they move from one party to another differently from the electorate.

In the olden days, when polling worked rather better, it was of course done by face-to-face samples, as the noble Lord, Lord Foulkes, has reminded us. Pollsters were able to get it quite right because they were able to balance their samples to make up for these shortfalls. The crucial thing was that in those days voting very closely followed social class: working-class people voted for us, while middle and upper-class people voted for the other side. As a result, it was relatively easy to adjust your sample. As long as you had the right percentage of people from each class in it, you were probably not going to be far out, and therefore you could weight your samples accordingly. However, that relationship between class and voting has gradually been eroded, and there is not really a good substitute. Public versus private sector, for example, is something of a differentiator, but there is nothing as powerful as class used to be. It is very much less easy to weight a sample to get a correct one, although responsible pollsters make valiant efforts.

Another problem is that a lot of people just do not answer when you approach them: only one in five people who are rung up answer, because they do not know whether it is a push-pollster, somebody trying to sell them double-glazing or a genuine pollster. You do not know anything about those people, so you have no idea whether your sample is representative of the population as a whole. There are big margins of error. This is not of course helped by my old profession—I used to write up polls myself and I am guilty of all the offences I am about to accuse others of—in which you are forced to exaggerate. Suppose Labour has gone one month from 36% to 34%. The correct report of that is that there is no change in Labour’s standing; it is the same number. The report you will read will very likely say, “Labour slumped last month following Ed Miliband’s unfortunate incident with a bacon sandwich”. It had not fallen and there is no evidence that it had; there is even less evidence that it was due to Ed’s bacon sandwich. So journalism is more than a little responsible. Of course the two egg each other on, because the pollsters know that the more that journalists exaggerate the story the more publicity it will get, and the journalists want that publicity too, so the necessary caveats simply do not appear.

Having said that, I shall tell the House about the remark recorded in the Library Note about the problems of the polls and about my noble friend’s Bill in particular. The chairman of ComRes, on which I addressed the House in the Moses Room the other day, said that the Bill,

“acts on a problem that doesn’t exist”.

Most of the pollsters have been very upfront and honest in admitting that there is a problem, but not, it appears, all. This matters because—let us face it—the last general election was mostly about whether it was a good or bad thing if we had a Labour-minority Government backed by, or in some relationship with, the SNP, not a separate though equally important debate about whether people really wanted a Tory-majority Government or something short of that. Did it affect the result? We will never know, but it was important.

I turn briefly to the question raised by my noble friend’s Bill about the regulation of polling. You cannot very often separate my noble friend and me on these or indeed many issues, but I have a slightly different emphasis from him on this point. The point made by the noble Lord, Lord Cooper, about the difficulties of statutory regulation was strong, as indeed were his points about the banning of polls during the run-up to an election, but there are problems with statutory regulation. My own experience of statutory regulation is that the people you are regulating spend the whole time trying to find their way around the regulations, instead of doing what you want them to do, which is to say, “We’ve got a problem and we must solve it”.

I am therefore broadly in favour of self-regulation. I sat for years on the council of the Advertising Standards Authority, which I believe to be a very effective self-regulator, though not everyone agrees. I felt that that was the right way of going about it because you have advertiser buy-in; if they had an adverse verdict from the ASA, they did not argue the toss but just withdrew the advert. Unfortunately, the existing self-regulation does not measure up. The rules of the British Polling Council, which in many ways is an admirable organisation, cover transparency only; they tell you that you have to put your samples and the questions you ask online, that sort of thing, but they do not actually cover substance at all. The Market Research Society does somewhat better; a section of its rules is about how questions should be framed so that they are neutral, but not all polling companies belong to the MRS. In the case of my own complaint against ComRes at the moment, I have to complain about the chairman as an individual rather than the company. The MRS rules are rather broad, and less specific than I would like.

So there is a shortfall in self-regulation, and this is one of the challenges that the polling industry now faces. Either it puts in place effective, wide-ranging and considered self-regulation, going beyond the very limited regulation imposed by the BPC, or it will be Foulkes. There is no doubt about it. It will not happen in this Parliament, I do not suppose, because this Government are quite well favoured by what happened when it went wrong last time, but some of us will harbour a desire for revenge. If the polling industry does not put itself right, we will put it right in a way that will be undesirable.

I have a final word to say about the banning of polls in the run-up to elections. I am afraid that here I am a feeble “don’t know”. I think that there is a case for and a case against, and although I lean towards the case against because I do not like interfering with free speech, the case for is also powerful and, as has been pointed out, other countries such as France and India already ban polls. I suggest that one or other House of this Parliament, in an appropriate committee, should look at the arguments for and against this. It would be a very useful analytic piece of work and would inform future policy-making in this field.

I very much hope that the House will give a Second Reading to this Bill this afternoon, because this is a subject that cannot be too much debated in your Lordships’ House as it is being in the polling industry. As I have said, I do not agree with every dot or comma of the Bill, but I believe that, unless the spirit of the Bill is taken on board by the polling industry, this kind of problem will arise time and again in future.