This article is from the source 'guardian' and was first published or seen on . It last changed over 40 days ago and won't be checked again for changes.

You can find the current article at its original source at http://www.theguardian.com/news/datablog/2016/mar/29/support-leaving-eu-referendum-overstated-polls-analysis-suggests

The article has changed 2 times. There is an RSS feed of changes available.

Version 0 Version 1
Support for leaving EU likely to be overstated in polls, analysis suggests Support for leaving EU likely to be overstated in polls, analysis suggests
(5 months later)
Telephone polls that show a double-digit lead for those who want to remain in the European Union are most likely to accurately reflect the thinking of the British public, according to an analysis by one of the only polling experts to correctly predict the result of last year’s general election.Telephone polls that show a double-digit lead for those who want to remain in the European Union are most likely to accurately reflect the thinking of the British public, according to an analysis by one of the only polling experts to correctly predict the result of last year’s general election.
Matt Singh of Number Cruncher Politics, working with James Kanagasooriam of polling company Populus, examined the discrepancy between phone and online surveys, and concluded that relying on internet polling overstates the level of support for those who want to leave.Matt Singh of Number Cruncher Politics, working with James Kanagasooriam of polling company Populus, examined the discrepancy between phone and online surveys, and concluded that relying on internet polling overstates the level of support for those who want to leave.
Related: Be a part of the Guardian's EU referendum coverage
Phone polls show a solid lead for the remain camp of about 15-20 percentage points when the report’s analysis was carried out in February, although the gap has dropped to closer to 10 points more recently. Online polls point to a very close contest.Phone polls show a solid lead for the remain camp of about 15-20 percentage points when the report’s analysis was carried out in February, although the gap has dropped to closer to 10 points more recently. Online polls point to a very close contest.
The two say that phone polls are more effective at compelling respondents to choose between the options on that will appear on the ballot paper – remain or leave – and more closely align with the demographic make-up and social attitudes of the electorate. Most online surveys explicitly include an extra option, don’t know.The two say that phone polls are more effective at compelling respondents to choose between the options on that will appear on the ballot paper – remain or leave – and more closely align with the demographic make-up and social attitudes of the electorate. Most online surveys explicitly include an extra option, don’t know.
They say: “As things now stand our assessment is that the true state of public opinion lies between the online and phone numbers, but closer to phone, approximately two-thirds of the way towards the leads phone polls are on average giving remain.”They say: “As things now stand our assessment is that the true state of public opinion lies between the online and phone numbers, but closer to phone, approximately two-thirds of the way towards the leads phone polls are on average giving remain.”
Other recent polling appeared to support Singh and Kanagasooriam’s thesis, although with a smaller gap. On Tuesday, an Ipsos Mori phone poll showed remain’s lead halved from 49% to 41%. However, after the results were adjusted based on those more likely to vote, remain’s lead shrunk to just two points (48% to 46%).Other recent polling appeared to support Singh and Kanagasooriam’s thesis, although with a smaller gap. On Tuesday, an Ipsos Mori phone poll showed remain’s lead halved from 49% to 41%. However, after the results were adjusted based on those more likely to vote, remain’s lead shrunk to just two points (48% to 46%).
However, a simple average of the latest EU referendum polls puts the vote to remain in the EU slightly ahead with 52% backing staying in compared to 48% for leave because online surveys say the race is tight.However, a simple average of the latest EU referendum polls puts the vote to remain in the EU slightly ahead with 52% backing staying in compared to 48% for leave because online surveys say the race is tight.
In the run up to the general election last year, polls overestimated the proportion of voters backing the Labour party. But Singh published a piece the day before the election publicly saying that he thought the Conservative vote might be underestimated.In the run up to the general election last year, polls overestimated the proportion of voters backing the Labour party. But Singh published a piece the day before the election publicly saying that he thought the Conservative vote might be underestimated.
A British Polling Council inquiry earlier this year found that sampling error to get a randomised sample of voters was largely to blame for the polling failure.A British Polling Council inquiry earlier this year found that sampling error to get a randomised sample of voters was largely to blame for the polling failure.
The analysis by the duo suggest a series of reasons to explain the gap between phone and online polling. The majority of online polls provide an explicit “don’t know” option. In most phone surveys voters have remain and leave as options, while are marked as undecided only if they say they are.The analysis by the duo suggest a series of reasons to explain the gap between phone and online polling. The majority of online polls provide an explicit “don’t know” option. In most phone surveys voters have remain and leave as options, while are marked as undecided only if they say they are.
Related: Let’s be brutally honest: this remain campaign is failing | Jackie Ashley
Voters are more likely to provide an opinion if they do not have a clear option to avoid doing so, Singh and Kanagasooriam conclude. They say this explains about a third of the difference in the results between the two methods of polling.Voters are more likely to provide an opinion if they do not have a clear option to avoid doing so, Singh and Kanagasooriam conclude. They say this explains about a third of the difference in the results between the two methods of polling.
The current methods used by pollsters to adjust the voter samples are also not considered by the report’s authors to be a good predictor of how people feel about the EU. The report says: “Simply put, people who seem otherwise alike in terms of who they are, where they live or how they’ve voted previously can turn out to be different on the question of the EU referendum depending on whether they are part of a phone or an online sample.”The current methods used by pollsters to adjust the voter samples are also not considered by the report’s authors to be a good predictor of how people feel about the EU. The report says: “Simply put, people who seem otherwise alike in terms of who they are, where they live or how they’ve voted previously can turn out to be different on the question of the EU referendum depending on whether they are part of a phone or an online sample.”
They argue that further adjusting results on the basis of social attitudes towards gender, race and national identity may give a more accurate picture of what Britain thinks.They argue that further adjusting results on the basis of social attitudes towards gender, race and national identity may give a more accurate picture of what Britain thinks.
Singh and Kanagasooriam also say that online polls underrepresent voters who are harder to reach – and that this particular group is more likely to have socially liberal opinions than the country as a whole. Conversely, online surveys overrepresent those who wear their political opinions on their sleeve; in this case, a group which is more likely to say they are in favour of leaving in the EU.Singh and Kanagasooriam also say that online polls underrepresent voters who are harder to reach – and that this particular group is more likely to have socially liberal opinions than the country as a whole. Conversely, online surveys overrepresent those who wear their political opinions on their sleeve; in this case, a group which is more likely to say they are in favour of leaving in the EU.
Data from the British Election Study (BES) shows that people likely to give their opinion the first time they are contacted are much more likely to be in favour of leaving the EU than those who require repeated attempts to reach them.Data from the British Election Study (BES) shows that people likely to give their opinion the first time they are contacted are much more likely to be in favour of leaving the EU than those who require repeated attempts to reach them.
Singh and Kanagasooriam warn that pollsters’ attempts to introduce “changes in methods are concentrating on trying to get the last election right rather than preparing for the very different kind of challenge posed by the EU referendum question.”Singh and Kanagasooriam warn that pollsters’ attempts to introduce “changes in methods are concentrating on trying to get the last election right rather than preparing for the very different kind of challenge posed by the EU referendum question.”
The risk, they write, is that any adjustment based on the past composition of an electorate would be less robust if the voters that turn out on 23 June are substantially different to those that voted last year.The risk, they write, is that any adjustment based on the past composition of an electorate would be less robust if the voters that turn out on 23 June are substantially different to those that voted last year.
In order to analyse and evaluate the two different methodologies, the authors relied on the post-election face-to-face polling carried out by the BES, which proved better than pre-election polls at interpreting how the nation voted. The BES data showed that 51% supported remaining in the EU while 33% wanted to leave. The BES survey also asked questions related to gender, racial equality and national identity.In order to analyse and evaluate the two different methodologies, the authors relied on the post-election face-to-face polling carried out by the BES, which proved better than pre-election polls at interpreting how the nation voted. The BES data showed that 51% supported remaining in the EU while 33% wanted to leave. The BES survey also asked questions related to gender, racial equality and national identity.
Related: Whether it likes it or not, Bank of England is in the thick of Brexit debate
The two authors also commissioned four new polls, two online and two on the phone, and split samples in order to test their assumptions. For example, half a sample was given a don’t know option, while another half wasn’t. This method also allowed to determine that those surveyed over the phone tend to be more socially liberal than those polled online. These ultimately produced the double digit lead for remain.The two authors also commissioned four new polls, two online and two on the phone, and split samples in order to test their assumptions. For example, half a sample was given a don’t know option, while another half wasn’t. This method also allowed to determine that those surveyed over the phone tend to be more socially liberal than those polled online. These ultimately produced the double digit lead for remain.
David Cameron has warned that low turnout could see Britain sleepwalk out of Europe. The report’s authors agree that turnout could be crucial to the final result: “Most of the polls we’ve looked at have not been weighted by likelihood to vote in a referendum, so turnout modelling is unlikely to be a factor in the discrepancy between the results of phone and online. That does not mean that turnout won’t be absolutely crucial to the ultimate result. The notion that leave voters are more determined to vote in the referendum, based on what leavers have been telling pollsters, has been little questioned.David Cameron has warned that low turnout could see Britain sleepwalk out of Europe. The report’s authors agree that turnout could be crucial to the final result: “Most of the polls we’ve looked at have not been weighted by likelihood to vote in a referendum, so turnout modelling is unlikely to be a factor in the discrepancy between the results of phone and online. That does not mean that turnout won’t be absolutely crucial to the ultimate result. The notion that leave voters are more determined to vote in the referendum, based on what leavers have been telling pollsters, has been little questioned.
“But this conclusion is heavily dependent on sampling people representatively in terms of relative political interest. Turnout self-reporting is known to be misleading this far out. If instead we consider high and low-turnout demographics, older people are more likely to vote to leave, but more affluent people are more likely to vote to remain than less affluent people.”“But this conclusion is heavily dependent on sampling people representatively in terms of relative political interest. Turnout self-reporting is known to be misleading this far out. If instead we consider high and low-turnout demographics, older people are more likely to vote to leave, but more affluent people are more likely to vote to remain than less affluent people.”