Effects of Filter Questions in Public Opinion Surveys

Abstract

Extending previous work, the authors find that the wording of a filter question can make a significant difference in the percentage of don't know (DK) responses elicited by an item, especially with topics that are more abstract or less familiar to survey respondents. They also find, however, that the content of an item can have a substantial, independent effect on DK or no responses, regardless of how the filter question is worded. In general, it appears that the less familiar the issue or topic, the greater the increase in DK responses produced by adding a filter. Even more important, the analysis shows that filtering can in some instances dramatically affect the conclusions a pollster would draw about the distribution of public opinion on an issue. Indeed, such effects may occur more often than has previously been suspected, though the circumstances under which they emerge remain elusive. The authors suggest that such effects may become amenable to analysis by probing respondents about what they had in mind as they answered the question. George F. Bishop is Associate Professor of Political Science and a Senior Research Associate at the Institute for Policy Research at the University of Cincinnati. Robert W. Oldendick is Assistant Director and Alfred J. Tuchfarber is Director of the Institute for Policy Research, University of Cincinnati. This research was supported by a grant from the National Science Foundation (SOC 78-07407). The authors want to thank the anonymous reviewers for their useful comments and suggestions for revising the original manuscript. Public Opinion Quarterly Vol. 47:528-546 ? 1983 by the Trustees of Columbia University Published by Elsevier Science Publishmg Co., Inc. 0033-362X/83/0047-528/$2.50 This content downloaded from 207.46.13.114 on Thu, 26 May 2016 06:44:05 UTC All use subject to http://about.jstor.org/terms EFFECTS OF FILTER QUESTIONS 529 about political and social issues, they found that a filter will generally increase the percentage of don't know (DK) or no responses to an item by about 20-25 percent.1 Furthermore, their analysis indicates that these increments in DK responses do not depend on the content of an issue, and they did not find any relationship between the percentage of DKs which were volunteered on an issue in the absence of an explicit filter question (the standard form) and the percentage of respondents removed by adding one. Yet they did discover that the wording of a filter can make a substantial difference in the percentage of respondents who say they have no opinion. A filter, that is, which emphasizes the frequency or acceptability of not having an opinion on an issue will screen out many more people than one which does not. Surprisingly, perhaps, their research also suggests that in most instances filtering will have little impact on the distribution of substantive responses to an item once the DKs are excluded from the analysis. The use of filter questions in their experiments, moreover, did not appear to have any significant influence on the magnitude of association between substantive responses to issues and such demographic variables as age, sex, and education. A researcher would, in other words, draw essentially the same conclusion about the nature and determinants of public opinion on an issue on the basis of either a filtered or an unfiltered form (see Schuman and Presser, 1981:126-28,