This is being debated vigorously in the polling and data analytics communities and that debate should and will continue for quite awhile. The New York Times has had some useful material about this, including a long article with historical background on polling by David Leonhardt, which I recommend. There are other pieces by the Nates (Cohn and Silver) and a good interview of David Shor by Dylan Matthews on Vox. But if you're going to read just one thing--and really, how many do you really want to read--I would suggest this short, crisp article by Scott Keeter, Courtney Kennedy and Claudia Deane on the Pew site. I declare it fair and balanced!
They march through various possibilities, clearly describing the possible problem and its possible remedies. They start with what I take to be the evolving consensus on the most important problem: partisan nonresponse:
"* The suggested problem
According to this theory, Democratic voters were more easily reachable and/or just more willing than Republican voters to respond to surveys, and routine statistical adjustments fell short in correcting for the problem. A variant of this: The overall share of Republicans in survey samples was roughly correct, but the samples underrepresented the most hard-core Trump supporters in the party. One possible corollary of this theory is that Republicans’ widespread lack of trust in institutions like the news media – which sponsors a great deal of polling – led some people to not want to participate in polls.
* Is this mainly an election polling problem, or would this be of wider concern to issue pollsters as well?
Sadly, the latter. If polls are systematically underrepresenting some types of conservatives or Republicans, it has ramifications for surveys that measure all kinds of behaviors and issues, from views on the coronavirus pandemic to attitudes toward climate change. Issue polling doesn’t require the kind of 51%-49% precision of modern presidential election polling, of course, but no pollster wants a systematic skew to their data, even if it’s “only” 5 percentage points.
* What could we do to fix it?
A straightforward fix to the problem of underrepresenting Trump supporters would be to increase efforts to recruit conservatives and Republicans to polls; increase the statistical weight of those already in the survey to match their share of the population (a process known as “weighting”); or both. Many polls this year weighted on party registration, 2016 vote or self-identified partisanship, but still underestimated GOP support.
The challenge here is twofold. The first is in estimating the correct share of conservatives and Republicans in the population, since, unlike age, gender and other demographic characteristics, there are no timely, authoritative benchmarks on political orientation. Second, just getting the overall share of Republicans in the poll correct may be insufficient if those who are willing to be interviewed are bad proxies for those who are not willing (e.g., more strongly conservative) – in which case a weighting adjustment within partisan groups may be needed."
If this doesn't sound like an easy problem to solve, that's because it isn't!
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.