John Halpin at The Liberal Patriot (subscribe, damn it--it's free!) has a terrific piece up on how polls can deceive and how we can do better. On the sampling problem:
"After 2020, data analyst David Shor looked into this exact issue offering an intriguing explanation of why polls didn’t include representative numbers of Trump voters—mainly that Trump-leaning voters don’t trust institutions like the media or non-profit organizations who sponsor most polls and therefore don’t even participate in surveys. Shor proffered that higher-trust Democrats were much more likely to take surveys during the pandemic lockdowns than were low-trust Republicans, thus skewing the samples of many polls. President Trump’s frequent admonitions against “fake news” and “fake polls” certainly fueled existing doubts among many of these voters. However, weighting by education alone does not solve the issue Shor raises since samples of non-college educated whites that don’t include enough low-trust, Trump-leaning cohorts will remain flawed."
But very importantly, he points out:
"As analysts and industry leaders work on fixes and corrections to ensure solid samples and better surveys, it’s worth considering another source of problems in the polling world—primarily, too many polls released today are basically propaganda and don’t offer any real understanding of how voters process dense social and economic topics or how they absorb competing understandings of what specific ideas and issues actually mean.
If you don’t get a representative sample of people in your poll, your poll will be bad. If you ask poor or misleading questions—or spin the narrow margins of the responses to these questions as showing “strong support” for your issue—your poll will also be bad.
Unfortunately, people spend less time on these problems of design and interpretation in polling than they do on the technical aspects of methodology. "
He has several suggestions for improvement, including this one:
"Challenge ideological assumptions going into any public opinion project. On top of excessive messaging, many projects suffer at the outset from an unwillingness or inability of the sponsors of the work to consider that voters may think differently than them on the issue or have a divergent interpretation of what their issue means and how it should be structured.
The point of good research is to find out political opportunities and challenges on an issue, not to reiterate views or build false self-confidence in ideas that an organization already holds.
This is apparent in a lot of good work on criminal justice reform. For example, research I’ve conducted on these reforms shows that concrete issues of discrimination—such as racial profiling of black men or excessive sentencing guidelines associated with the war on drugs—grounded in values like fairness and equal dignity for all people generate more concern and willingness to act among Americans than do abstract indictments of “systemic racism” across the entire criminal justice sphere.
Shifting public sentiments on BLM protests throughout the past summer also highlight the complexity of opinion about these issues that a purely social justice approach to them might miss. What started with widespread condemnation of police behavior and racial discrimination in the immediate aftermath George Floyd’s death, along with massive public support for criminal justice reforms, rapidly dissolved over the summer as the decentralized movement shifted to more radical protests and indictments of structural racism, wider decriminalization, and issues like reparations or defunding the police that have little to no public support. If movements want to be successful, and pass laws rather than just engage in sloganeering, they need to accept and understand people as they exist, warts and all.
Not surprisingly, most of the early Biden moves on these issues now center on consensus ideas with more public backing built around police training, sentencing fairness, eliminating discrimination, offering second chances, and reducing violence."
Read it all at The Liberal Patriot!
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.