This article is from the source 'nytimes' and was first published or seen on . It last changed over 40 days ago and won't be checked again for changes.
You can find the current article at its original source at http://www.nytimes.com/2016/09/29/us/politics/why-you-shouldnt-believe-most-online-polls.html
The article has changed 5 times. There is an RSS feed of changes available.
| Version 0 | Version 1 |
|---|---|
| Why You Shouldn’t Trust ‘Polls’ Conducted Online | |
| (about 1 hour later) | |
| On Tuesday morning, Donald J. Trump boasted of what he described as widespread approval of his performance in the first presidential debate with Hillary Clinton. | On Tuesday morning, Donald J. Trump boasted of what he described as widespread approval of his performance in the first presidential debate with Hillary Clinton. |
| Those so-called “final debate polls” consisted of readers on news sites who were asked for their opinion of who had won. | Those so-called “final debate polls” consisted of readers on news sites who were asked for their opinion of who had won. |
| While it looked good for Mr. Trump, pollsters and some journalists offered a protest: Informal polls on news sites tend to produce junk data that does not indicate how the public actually feels, and should not be believed as an indication of — well, much of anything. | While it looked good for Mr. Trump, pollsters and some journalists offered a protest: Informal polls on news sites tend to produce junk data that does not indicate how the public actually feels, and should not be believed as an indication of — well, much of anything. |
| “Those do a good job of engaging audiences online, and they do a good job of letting you know how other people who have come to the webpage feel about whatever issue,” said Mollyann Brodie, the executive director for public opinion and survey research at the Kaiser Family Foundation. “But they’re not necessarily good at telling you, in general, what people think, because we don’t know who’s come to that website and who’s taken it.” | “Those do a good job of engaging audiences online, and they do a good job of letting you know how other people who have come to the webpage feel about whatever issue,” said Mollyann Brodie, the executive director for public opinion and survey research at the Kaiser Family Foundation. “But they’re not necessarily good at telling you, in general, what people think, because we don’t know who’s come to that website and who’s taken it.” |
| Professional pollsters must ensure that their small samples are demographically appropriate to indicate how larger groups of people think. Online polls do nothing of the sort, allowing anyone who finds the poll to vote, thus opening them to manipulation from those who would want to stuff the ballot box. Users on Reddit and 4chan directed masses of people to vote for Mr. Trump in the instant-analysis polls, according to The Daily Dot. | Professional pollsters must ensure that their small samples are demographically appropriate to indicate how larger groups of people think. Online polls do nothing of the sort, allowing anyone who finds the poll to vote, thus opening them to manipulation from those who would want to stuff the ballot box. Users on Reddit and 4chan directed masses of people to vote for Mr. Trump in the instant-analysis polls, according to The Daily Dot. |
| Even when there is no intentional manipulation, the results are largely a reflection of who is likely to come to a site and who would be motivated enough to participate in a poll. Intuitively, it’s no surprise that voters on sites like Breitbart News and the Drudge Report would see Mr. Trump as the winner, just as Mrs. Clinton would be more likely to find support on liberal sites. | Even when there is no intentional manipulation, the results are largely a reflection of who is likely to come to a site and who would be motivated enough to participate in a poll. Intuitively, it’s no surprise that voters on sites like Breitbart News and the Drudge Report would see Mr. Trump as the winner, just as Mrs. Clinton would be more likely to find support on liberal sites. |
| It’s not just in politics; take the question of identifying the best undefeated team in the N.F.L. after three weeks. The New England Patriots and the Denver Broncos are likely the most obvious candidates for football fans, but when a news site in Philadelphia asked the question on Sunday after a big win by the Eagles? You can guess what happened. | It’s not just in politics; take the question of identifying the best undefeated team in the N.F.L. after three weeks. The New England Patriots and the Denver Broncos are likely the most obvious candidates for football fans, but when a news site in Philadelphia asked the question on Sunday after a big win by the Eagles? You can guess what happened. |
| While that’s an extreme case of hometown bias, similar factors are at play on every website. Frank Newport, the editor in chief of Gallup, said most reader polls on news sites are done more to benefit the news organization than to educate the public. | While that’s an extreme case of hometown bias, similar factors are at play on every website. Frank Newport, the editor in chief of Gallup, said most reader polls on news sites are done more to benefit the news organization than to educate the public. |
| “In our business, the key is generalizability,” he said, referring to the ability of a sample group to apply to a wider population. “That’s the core of what we do. Typically, it takes a lot of time, and a lot of effort, and a lot of money to do it.” | “In our business, the key is generalizability,” he said, referring to the ability of a sample group to apply to a wider population. “That’s the core of what we do. Typically, it takes a lot of time, and a lot of effort, and a lot of money to do it.” |
| The New York Times conducts scientific polling and does not feature unscientific reader polls on its website. The New York Times Magazine asks a selection of its readers offbeat questions, but each edition discloses the unscientific methodology. | The New York Times conducts scientific polling and does not feature unscientific reader polls on its website. The New York Times Magazine asks a selection of its readers offbeat questions, but each edition discloses the unscientific methodology. |
| Andrew Seaman, the ethics committee chairman for the Society of Professional Journalists, said the common use of online opt-in polling warranted a review in newsrooms. | Andrew Seaman, the ethics committee chairman for the Society of Professional Journalists, said the common use of online opt-in polling warranted a review in newsrooms. |
| Even polls with little scientific rigor could have a big impact, he said. In a hypothetical situation, he imagined a local news site that crowned the best pizza shop based on 13 responses in a city of 10,000 people. It might seem benign, but other pizza shop owners could lose business — and the stakes would be higher in more significant cases, like a presidential election. | Even polls with little scientific rigor could have a big impact, he said. In a hypothetical situation, he imagined a local news site that crowned the best pizza shop based on 13 responses in a city of 10,000 people. It might seem benign, but other pizza shop owners could lose business — and the stakes would be higher in more significant cases, like a presidential election. |
| “My advice would be to avoid opt-in polls that provide little value to the public,” Mr. Seaman said in an email. “Otherwise, the news organizations run the risk of being irresponsible and possibly harmful.” | “My advice would be to avoid opt-in polls that provide little value to the public,” Mr. Seaman said in an email. “Otherwise, the news organizations run the risk of being irresponsible and possibly harmful.” |