Tuesday 19 January 2016

Polling

From the BBC

The failure of pollsters to forecast the outcome of the general election was largely due to "unrepresentative" poll samples, an inquiry has found.
The polling industry came under fire for predicting a virtual dead heat when the Conservatives ultimately went on to outpoll Labour by 36.9% to 30.4%.
A panel of experts has concluded this was due to Tory voters being under-represented in phone and online polls.
But it said it was impossible to say whether "late swing" was also a factor.

One of the problems I think with polling is how do you know when you have a cross-section of the population? You can probably write down some criteria, but are those right? And even within those criteria, what is the effect of the sort of people that do surveys vs the sort of people who don't? I don't do them for once simple reason: they don't pay me enough. I'm not going to talk to someone on the phone for 20 minutes for the chance of winning £250 of M&S vouchers where they won't tell me how many other people go into that draw. If it's 50 people, I'd be interested. But if it was an average £5 outcome, they'd just send me a voucher. So I think it's probably more like £1. Does it mean people who understand the psychology of competitions and probability don't do surveys, and what effect does that have?

As for "late swing", it's not "late swing". It's about stated vs received preference (virtue signalling) and people thinking harder about something when it has a cost. I've organised work trips to things and everyone's really enthusiastic until you ask for payment, then some people get a bit sheepish. The money forces them to go from "would this be awesome" to "and is it worth the money". The polling industry thought they had this nailed after 97, but I think it was more that the public didn't really see Blair as much of a socialist, much of a cost, so there wasn't much of a difference.

12 comments:

Rich Tee said...

I remember one pollster's comment: "It's difficult to get people to answer the phone nowadays."

That will be because people are fed up of being bombarded with nuisance calls and so don't answer anonymous or unexpected calls anymore. I dare say it is older people at home who are more likely to be like this.

SumoKing said...

Is this even a thing?

I mean this was a top 2/3 news story this morning and really "guestimates of who out of basically 2 guys would win the election were a bit out and this justified some sort of report" isn't really news. It's more of an embarrassed media circle jerk.

Do these polls even matter to anyone? Or are they just bell weathers for strategists and the decrepit pundits that nobody under the age of about 500 has any interest in listening to?

Curtis said...

I do lots of online surveys. I always answer Labour, even though I despise the Labour party. Don't know if there are any others like me or which direction they mislead pollsters in.

L fairfax said...

Weirdly the exit poll was right whilst in 1992 all the polls including the exit poll were equally wrong.

Tim Almond said...

Rich Tee,
I'd get rid of mine except it's part of my broadband deal, and my mother still uses it. Everyone else calls my mobile or emails me.

SumoKing,
My own points were just that this sort of thing, of predicting human behaviour and psychology of it, kinda interest me, and yes, I believe that polling and focus groups only work if you ask people very general things like priorities or similar cost alternatives. The Guardian fell foul of this - they announced Guardian Membership and asked a room of people if they'd join and nearly everyone put their hand up. If Alan Rusbridger had then gone round the room with subscription forms, he'd have found out the real answer.

Curtis,
I love these forms on websites that want answers to proceed, like I'm going to do anything but click any answer to move on.

L Fairfax,
And what happened in that poll? You had a higher taxing Labour party - there was an extra cost. It was only as people walked to the booth that they had to discard the "wouldn't it be nice if..." for "what do I want".

I still think that Donald Trump might fail for this reason, that Trump is the fantasy solution rather than the real-world one. It's like people who say they like maverick politicians. It's only because they're being entertained by mavericks. Ask someone to vote for them as a potential prime minister, someone who's going to make sure their schools are good and the roads get fixed and they'll run a mile.

L fairfax said...

@"L Fairfax,
And what happened in that poll? You had a higher taxing Labour party - there was an extra cost. It was only as people walked to the booth that they had to discard the "wouldn't it be nice if..." for "what do I want"."
But in 92 the exit poll was wrong which can't have been because people changed their minds (it was an exit not an entrance poll).

Tim Almond said...

L Fairfax,

The exit poll of 92 wasn't far off. It predicted Conservatives short by 25 and instead were +21. But you have to remember, until that poll, all other polls had been predicting a Labour win with a majority around 19-23. The exit poll was certainly considerably more accurate than the previous polls by quite some margin, and ever since, exit polls have been pretty solid.

L fairfax said...

@The Stigler
In that case my memory must be wrong, thanks for the clarification.

DBC Reed said...

Outlaw all such polls (if this is possible).The last set influenced the result because they were all predicting coalitions, with Labour hooked up with the much more radical SNP which the BBC especially ran as a huge Red Scare story .The Conservatives were hooked up in the projected coalitions scenario with the Lib Dems as per usual who being useless were not perceived as much of a threat.

Mark Wadsworth said...

Agreed.

Anonymous said...

on the other hand...Nate Silver correctly predicted every single state correctly in the 2012 US presidential election.

Tim Almond said...

paulc156,

True. Nate works by trying to look at what got the answer right before, which makes some sense. So, it's take some polling and tweak it.

He tried to have a bash at predicting Oscar wins and did very well, by measuring how much other awards matched historically (so for example, the Directors Guild of America regularly came up with similar winners for Best Director, SAG awards regularly for best actress etc) and got 5/6 of the big ones, and even with the one he got wrong last year (Richard Linklater for best director), it was a close call between him and Inarritu.