This article is from the source 'guardian' and was first published or seen on . It last changed over 40 days ago and won't be checked again for changes.
You can find the current article at its original source at http://www.theguardian.com/us-news/datablog/2015/dec/06/trump-factor-election-polling-debate-accuracy-value
The article has changed 5 times. There is an RSS feed of changes available.
| Version 3 | Version 4 |
|---|---|
| Trump factor fuels debate about value of 'horse race' polls in election cycle | Trump factor fuels debate about value of 'horse race' polls in election cycle |
| (about 17 hours later) | |
| An American presidential election is perhaps the world’s ultimate opinion poll: 235 million participants, billions of dollars and 18 months of fierce political debate. | An American presidential election is perhaps the world’s ultimate opinion poll: 235 million participants, billions of dollars and 18 months of fierce political debate. |
| Related: Jeb Bush puts trust in God as polling plummets and purse strings tighten | Related: Jeb Bush puts trust in God as polling plummets and purse strings tighten |
| Yet this extravagant exercise in democracy is increasingly overshadowed by a cheaper variety of polling, one that often relies on an anonymous, self-selecting sample of online respondents numbering as few as 500 to serve as a proxy for the entire electorate. | Yet this extravagant exercise in democracy is increasingly overshadowed by a cheaper variety of polling, one that often relies on an anonymous, self-selecting sample of online respondents numbering as few as 500 to serve as a proxy for the entire electorate. |
| Rarely has this sampling methodology been as much in question, but in the celebrity-fueled 2016 election cycle, many experts fear that the perception of success is now everything; what should just be an indication of voting intentions is becoming the determining factor. | Rarely has this sampling methodology been as much in question, but in the celebrity-fueled 2016 election cycle, many experts fear that the perception of success is now everything; what should just be an indication of voting intentions is becoming the determining factor. |
| Because polls are used to decide who appears in official television debates, lesser-known candidates are starved of the chance to introduce themselves while donors flock toward “winners” and a media echo chamber amplifies only the loudest. | Because polls are used to decide who appears in official television debates, lesser-known candidates are starved of the chance to introduce themselves while donors flock toward “winners” and a media echo chamber amplifies only the loudest. |
| “It’s become an election about polls, and that’s never a healthy state of affairs,” says one Republican campaign manager. | “It’s become an election about polls, and that’s never a healthy state of affairs,” says one Republican campaign manager. |
| In recent days, debate has raged most fiercely around Donald Trump. One group of pundits point to exaggerated polling as the explanation for his temporarily dominant lead in the Republican primary race. Other commentators suggest this ignores genuine momentum behind his campaign that has lasted months longer than anyone predicted. | In recent days, debate has raged most fiercely around Donald Trump. One group of pundits point to exaggerated polling as the explanation for his temporarily dominant lead in the Republican primary race. Other commentators suggest this ignores genuine momentum behind his campaign that has lasted months longer than anyone predicted. |
| But an alternative explanation is also emerging. What if both views are right? What if the polling is flawed and Trump still stays ahead, in a self-fulfilling prophecy with profound consequences for the effectiveness of the democratic process? | But an alternative explanation is also emerging. What if both views are right? What if the polling is flawed and Trump still stays ahead, in a self-fulfilling prophecy with profound consequences for the effectiveness of the democratic process? |
| There is certainly a precedent internationally for polls that have proved wildly inaccurate and hugely influential at the same time. Pollsters underestimated Republican support ahead of the the 2014 midterms, causing the White House to delay immigration reform in the mistaken belief that Democrats had a chance of holding on to marginal states. | There is certainly a precedent internationally for polls that have proved wildly inaccurate and hugely influential at the same time. Pollsters underestimated Republican support ahead of the the 2014 midterms, causing the White House to delay immigration reform in the mistaken belief that Democrats had a chance of holding on to marginal states. |
| Similarly, polls incorrectly said the Scottish referendum was too close to call, prompting the controversial intervention of the Queen and lasting bitterness among many Scots. | Similarly, polls incorrectly said the Scottish referendum was too close to call, prompting the controversial intervention of the Queen and lasting bitterness among many Scots. |
| Surveys said the 2015 Israeli national election would be an incredibly tight race. It wasn’t. Two months later, they said the same about the UK general election. They were wrong again, but it did not stop each prediction causing huge waves. | Surveys said the 2015 Israeli national election would be an incredibly tight race. It wasn’t. Two months later, they said the same about the UK general election. They were wrong again, but it did not stop each prediction causing huge waves. |
| Increasingly, reputable pollsters are pulling out of so-called “horse race” election polling because they fear it risks tarnishing their brand. | Increasingly, reputable pollsters are pulling out of so-called “horse race” election polling because they fear it risks tarnishing their brand. |
| “When you have lots of candidates running and you are months ahead of the first voting and you ask people who they are going to vote for, you are just basically getting top of mind impressions and, to a large degree, name recognition,” says Gallup’s Frank Newport. | “When you have lots of candidates running and you are months ahead of the first voting and you ask people who they are going to vote for, you are just basically getting top of mind impressions and, to a large degree, name recognition,” says Gallup’s Frank Newport. |
| Gallup bows out | Gallup bows out |
| Accused of badly misreading the 2012 US election, Gallup has given up entirely on the horse race polls for the 2016 primary and may not even conduct them ahead of November’s general election. | Accused of badly misreading the 2012 US election, Gallup has given up entirely on the horse race polls for the 2016 primary and may not even conduct them ahead of November’s general election. |
| “It will be much less a part of what we are doing than in 2012 when we were tracking the horse race every night; we certainly will not be doing that this election,” reveals Newport. Rather, Gallup will move towards more in-depth polling, touching on the “fundamentals and issues”. | “It will be much less a part of what we are doing than in 2012 when we were tracking the horse race every night; we certainly will not be doing that this election,” reveals Newport. Rather, Gallup will move towards more in-depth polling, touching on the “fundamentals and issues”. |
| It may seem like a cop-out, but these less-direct polling methods, such as tracking favourability, show dramatically different results – with Marco Rubio, for example, achieving Gallup net favourability ratings more than double that of the supposed frontrunner Trump. | It may seem like a cop-out, but these less-direct polling methods, such as tracking favourability, show dramatically different results – with Marco Rubio, for example, achieving Gallup net favourability ratings more than double that of the supposed frontrunner Trump. |
| Other pollsters believe they have no choice but to continue asking the more straightforward question of voting intentions – even if the answer is less and less reliable. | Other pollsters believe they have no choice but to continue asking the more straightforward question of voting intentions – even if the answer is less and less reliable. |
| One executive with the British-based YouGov says its failures during the last UK election had a disquieting effect on internal morale but strangely little impact on external clients, perhaps because the whole industry got it wrong and the pain was shared so widely. | One executive with the British-based YouGov says its failures during the last UK election had a disquieting effect on internal morale but strangely little impact on external clients, perhaps because the whole industry got it wrong and the pain was shared so widely. |
| And while the polling industry may be in the grip of an existential crisis of confidence, the US media and political establishment has never relied on it more heavily. | And while the polling industry may be in the grip of an existential crisis of confidence, the US media and political establishment has never relied on it more heavily. |
| One Republican primary candidate, South Carolina senator Lindsey Graham, was relegated first from the main TV debate stage and then even from the secondary presidential debate on the basis of a difference in polls well within the statistical margin of error, and one of which did not even mention the senator by name. | One Republican primary candidate, South Carolina senator Lindsey Graham, was relegated first from the main TV debate stage and then even from the secondary presidential debate on the basis of a difference in polls well within the statistical margin of error, and one of which did not even mention the senator by name. |
| Graham’s campaign manager, Christian Ferry, who also worked for Senator John McCain in previous election cycles, says the problem is much worse this time around and is in danger of undermining the important role of voters in early primary states. | Graham’s campaign manager, Christian Ferry, who also worked for Senator John McCain in previous election cycles, says the problem is much worse this time around and is in danger of undermining the important role of voters in early primary states. |
| “Campaigns have traditionally been about how many people can you meet and how well you can organise in states like New Hampshire and Iowa,” Ferry says. “Senator McCain was not a well known figure at the outset in 2000, but there was a lot less focus on national polls. He started to see a surge in New Hampshire.” | “Campaigns have traditionally been about how many people can you meet and how well you can organise in states like New Hampshire and Iowa,” Ferry says. “Senator McCain was not a well known figure at the outset in 2000, but there was a lot less focus on national polls. He started to see a surge in New Hampshire.” |
| Though some may argue these states have had undue influence in the past, Ferry argues that turning to national polling for answers is a dangerous way to solve the problem. | Though some may argue these states have had undue influence in the past, Ferry argues that turning to national polling for answers is a dangerous way to solve the problem. |
| “If you have a problem with the early state primary system you need to change it, not try to compensate for it,” he adds. “Iowa, New Hampshire and South Carolina are small and have relatively inexpensive media markets and voters are very used to the process; they take their responsibility seriously. Otherwise we are talking about nationalising the election, which emphasises money and celebrity.” | “If you have a problem with the early state primary system you need to change it, not try to compensate for it,” he adds. “Iowa, New Hampshire and South Carolina are small and have relatively inexpensive media markets and voters are very used to the process; they take their responsibility seriously. Otherwise we are talking about nationalising the election, which emphasises money and celebrity.” |
| No easy fix | No easy fix |
| Some might dismiss the complaints of marginal campaigns like Graham’s as sour grapes, but increasingly the scepticism of polling is shared even by those for whom the results and the debate system have been kind. | |
| “We told our supporters in April when we were leading in the polls not to pay any attention to them and we told our supporters in August when we were lagging not to pay any attention to them,” says Marco Rubio’s spokesman Alex Conant. | “We told our supporters in April when we were leading in the polls not to pay any attention to them and we told our supporters in August when we were lagging not to pay any attention to them,” says Marco Rubio’s spokesman Alex Conant. |
| “After every debate we have seen a surge in fundraising, and volunteer sign-up and larger crowds and improved poll numbers, but of all those things the one we care least about is the improved poll numbers – because we believe they will go up and down a lot over the next eight weeks and the only time they are really going to matter at all is in February.” | “After every debate we have seen a surge in fundraising, and volunteer sign-up and larger crowds and improved poll numbers, but of all those things the one we care least about is the improved poll numbers – because we believe they will go up and down a lot over the next eight weeks and the only time they are really going to matter at all is in February.” |
| And the problems of polling are not easily fixed or ignored. Shying away from horse race predictions, as Gallup is, may only exacerbate the collapse of confidence in the industry. | And the problems of polling are not easily fixed or ignored. Shying away from horse race predictions, as Gallup is, may only exacerbate the collapse of confidence in the industry. |
| Elections offer a unique opportunity for polling companies. When surveys are conducted on Pope Francis, pollution or parenthood, respondents are stating opinions – their claims are no more verifiable than the numbers they produce. But at elections, once the votes are tallied, pollsters can use their numbers to say “Ta-da!” or “Told you so!” because they asked a nationally representative sample of individuals what the nation would do – and the nation did it. | |
| Related: What will it take to stop Donald Trump? | Related: What will it take to stop Donald Trump? |
| That nationally representative sample is becoming increasingly elusive though. First, a lot of Americans have ditched the landline phones that pollsters use to contact them. In 2003, less than 5% of adults owned a cellphone but had no landline. By the first half of this year, that figure had risen to 47%. The individuals who don’t have a landline are much more likely to be poorer, younger and non-white – they’re still potential voters though. And, since the law prohibits pollsters from auto-dialing Americans on their cellphones, that poses a real problem for polling companies. | |
| The decline of the landline has contributed to the growing gap between results found in online polls and those found in phone polls. In the past few months, Trump has been around five percentage points more popular in online polls than phone polls. Those internet polls are new and relatively untested which makes it difficult to gauge whether they will fare any better than phone polls in predicting elections. And while it’s true that respondents may be more willing to be honest when faced with a computer screen rather than a human voice, pollsters are never able to perfectly correct for one flaw in human behaviour that affects all these statistics – whether out of pride, shame or sheer laziness: people lie. | |
| Even if individuals can be contacted, increasingly, they refuse to speak. According to the American Association for Public Research, “response rates across all modes of survey administration have declined, in some cases precipitously”. In 1997, 36% of people who had been contacted would respond – by 2012, it was 9%. | Even if individuals can be contacted, increasingly, they refuse to speak. According to the American Association for Public Research, “response rates across all modes of survey administration have declined, in some cases precipitously”. In 1997, 36% of people who had been contacted would respond – by 2012, it was 9%. |
| Pollsters claim that this doesn’t have to affect their accuracy because they can use a technique called weighting. Let’s say you want to know what percentage of people in America will drink alcohol this weekend and so you ask ten people what their plans are. If four of the people that you talk to are children, you would give their answers less weight. That’s because nationally, only about two out of every 10 Americans is a child, and because the alcohol consumption habits of children and adults are very different. | |
| Weighting can improve the accuracy of the numbers – if done correctly. It’s important to understand if and how views vary between different groups. And it’s important that you’re still polling enough individuals before you apply any weightings. Take, for example, a recent survey by YouGov on faith – out of 999 respondents, only 72 said that they were Hispanic. Nationally though, 17% of the population is Hispanic. That might not sound like much, but when you take into account the fact that views also differ based on things like sex, age and income level in some cases, you’re left with less than ten individuals whose opinions need to be representative of thousands. Perhaps more worryingly, pollsters often use complex equations to do those weightings – equations which are not publicly available or verifiable. | Weighting can improve the accuracy of the numbers – if done correctly. It’s important to understand if and how views vary between different groups. And it’s important that you’re still polling enough individuals before you apply any weightings. Take, for example, a recent survey by YouGov on faith – out of 999 respondents, only 72 said that they were Hispanic. Nationally though, 17% of the population is Hispanic. That might not sound like much, but when you take into account the fact that views also differ based on things like sex, age and income level in some cases, you’re left with less than ten individuals whose opinions need to be representative of thousands. Perhaps more worryingly, pollsters often use complex equations to do those weightings – equations which are not publicly available or verifiable. |
| But reliance on polls barely seems to have changed. It seems that for now, however flawed the numbers might be, we have few alternatives to try and make sense of what will happen in November 2016. | But reliance on polls barely seems to have changed. It seems that for now, however flawed the numbers might be, we have few alternatives to try and make sense of what will happen in November 2016. |
| Some industry insiders believe the risk that false numbers do serious harm to the eventual result is overstated. | Some industry insiders believe the risk that false numbers do serious harm to the eventual result is overstated. |
| “It’s OK to have that feedback loop,” says Gallup’s Newport. “ I have a lot of faith in the average American to be able to sort through the information.” | |
| In particular he warns of the danger to free speech if the US were to go down the route of some countries and begin limiting the use of polling ahead of elections. | In particular he warns of the danger to free speech if the US were to go down the route of some countries and begin limiting the use of polling ahead of elections. |
| But perhaps, like the uncertainty principle in physics, the more an election is scrutinised the more the scrutiny risks interfering. If this interference is compounded by technological shifts making traditional poll methods increasingly unreliable, there could be a far larger threat to the democratic process than simply the free speech of pollsters. | But perhaps, like the uncertainty principle in physics, the more an election is scrutinised the more the scrutiny risks interfering. If this interference is compounded by technological shifts making traditional poll methods increasingly unreliable, there could be a far larger threat to the democratic process than simply the free speech of pollsters. |