11 December 2019

How much can we trust the exit poll?

By

This election has been billed as one of the most unpredictable ever, and not unreasonably – it’s harder than usual to say with any confidence what will happen. But one thing we can confidently predict is that after Big Ben strikes 10, and the broadcasters’ exit poll is revealed, someone will question its results.

The fact that the people downplaying the exit poll tend to be supporters of whichever party the poll shows doing badly, may or may not be indicative of their motivations for doing so. But how much can we trust exit polls?

The answer lies in how modern exit polls are conducted in the UK, and how they’re different from conventional polling. The sort of polls we see most of the time are done in less-than-ideal conditions for survey research, giving rise to a number of potential sources of error.

Firstly, the sample people you reach may not be representative of the population. Secondly, those within the sample that you expect to vote may not be the ones who actually do so. Thirdly, they may not be telling you the truth. And fourthly, they may change their minds – of if they’re undecided they may make up their minds in an unanticipated way – on the way to the polling station, particularly if you’ve spoken to them days beforehand.

Conducting an exit poll is a very different exercise. By interviewing every nth person leaving a polling station, most of the aforementioned risks are removed or reduced. Crucially, you have the holy grail of polling – a random sample. Some people may refuse to take part, but far fewer than would refuse a telephone poll, or even a door-to-door survey.

Additionally, someone leaving a polling station will have actually voted. By contrast, someone polled beforehand, even if they say they are absolutely certain to vote on the day, may fail to do so.

People telling fibs to pollsters is generally far less of a problem than many assume, but it can happen. So exit polls simulate the process of actually voting – the voter is given a mock ballot paper similar to the one they’ve just used to cast their real vote, and casts their exit poll “vote” in a similar way.

And finally, having actually voted, people cannot change their minds, and probably won’t have forgotten which party they voted for in the space of a minute or so – something which, for very different reasons, is currently a headache for us conventional pollsters.

The key difference, therefore, between exit polls and conventional polls is that the former have a significant advantage when it comes to the quality of the raw data. They do, however, have some additional challenges.

Most obviously, they only interview people voting on the day. With over 20% votes of being cast by post, and by a set of people who are very different to those voting in person, exit poll data contains a significant information gap.

The exit pollsters’ solution to this is remarkably simple, yet so far highly effective. Unlike the exit polls in many other countries, which ask a number of different questions, the UK exit poll only asks how people voted. This means that, since they are conducted at largely the same set of polling stations each time, the exit poll team can look at the change in the vote share estimates from the previous election, rather than the absolute levels.

This allows errors, whether from postal votes or anything else, to cancel each other out. The real risk therefore is a new source of error. That is quite plausible – after all, it’s hard to say what the changing nature of politics and voting patterns will throw up. But at least it doesn’t make the poll’s accuracy dependent on getting the absolute level right.

It’s for these reasons, together with skill of the psephologists and statisticians that model the individual seat results from the high quality raw data, that recent exit polls in the UK have such a strong track record.

The largest error on the seat total of the largest party since the current methodology has been in use was 15. This, ironically, came in 2015 – the year when the exit poll received more plaudits than any other – because the poll proved its value compared with the pre-election polls, which missed the significant Tory lead that ultimately gave David Cameron a majority.

So, while healthy scepticism of exit polls is reasonable up to a point, just as it is with conventional polls, the current specification of exit polls in the UK have an excellent record – precisely because the way they are done is very different.

Click here to subscribe to our daily briefing – the best pieces from CapX and across the web.

CapX depends on the generosity of its readers. If you value what we do, please consider making a donation.

Matt Singh is the founder of Number Cruncher Analytics