Measuring Opinions vs. Non-Opinions - The Case of the USA Patriot Act

Abstract

The public polling industry frequently faces the challenge of measuring and reporting opinions on topical policy issues about which the public may be uninformed or confused. These situations present a true challenge to pollsters who, through the process of polling itself, could end up manufacturing, rather than measuring, public opinion. Multiple studies have addressed the problems with manufacturing opinions on fictional issues, but none have addressed how or whether these effects occur in the real and possibly more complicated world of actual policy opinion. This paper uses multiple experiments to demonstrate how reported opinions on an actual issue on which the public is largely uninformed – the USA Patriot Act – vary greatly due to simple variations in question wording, content, and response options due to a lack of public knowledge about the Act. Because an uninformed public resorts to scouring questions for information cues with which they can formulate an answer, different question wordings provide different contexts and considerations for respondents and can thereby result in disparate answers to substantively similar questions.