Effects of Question Order on Survey Responses

Abstract

Two question orders were used in a Kentucky statewide random survey. In nonoverlapping sections, four questions which asked the respondents to express their general interests in politics and religion and their general evaluations of the gravity of the economic and energy situations either preceded or followed series of specific questions on the same issues. The respondents expressed significantly greater interest in politics and religion when these general questions followed the specific questions, but evaluations of the economic and energy crises were not significantly affected by question placement. Implications for the arrangement of questions within surveys are discussed. Sam G. McFarland is a Professor in the Department of Psychology at Western Kentucky University. This study was conducted as a part of a regular semiannual survey by the Kentucky Center for Survey Research and Services. The author wishes to thank Tom Madren and John Peterson, the codirectors of the center, for facilitating this research, and Howard Schuman and Fred Heilizer, for their comments on an earlier draft of this paper. Public Opinion Quarterly Vol. 45:208-215 ? 1981 by The Trustees of Columbia University Published by Elsevier North-Holland, Inc. 0033-362X/81/0045-208/$2.50 This content downloaded from 157.55.39.96 on Sat, 03 Sep 2016 05:49:34 UTC All use subject to http://about.jstor.org/terms EFFECTS OF QUESTION ORDER 209 more likely to view the federal tax burden as right rather than too high when the question followed the evaluations of federal spending in 11 specific areas (cited in Turner and Krauss, 1978). Other studies sponsored by NORC have failed to show the predicted order effects (Sudman and Bradburn, 1974). Bradburn and Mason (1974) proposed four specific types of possible order effects due to saliency, redundancy, consistency, and fatigue, but their own research found no order effects. Delamater and MacCorquodale (1975) found that responses to a series of questions on the respondents' sexual behaviors were not substantially affected by the placement of the series within a longer questionniare. Turner and Krauss (1978) have argued, however, that many of the significant differences in public confidence in national leaders found by different surveys are best explained by the differing contexts in which the questions appeared within the surveys. One common recommendation for survey construction is that questions which ask for a general evaluation on a particular issue should precede questions about its more specific aspects (cf. Komhauser and Sheatsley, 1976). This recommendation is apparently based on the untested assumption that the specific questions create a saliency or a specific set which might affect answers to the more general questions, whereas the general questions are less likely to influence responses on more specific subissues. This assumption served as the principal hypothesis for the present study. Because of the increased saliency of the issues, respondents were expected to express greater interest in religion and politics and greater concern about the economic and energy crises when a general question on each of these issues followed rather than preceded a series of special questions on the same topic. The magnitude of these order effects, if they occur, may also vary as a function of the demographics of the respondents. For example, respondents with little education may be more susceptible to order effects than are the more educated. As a consequence, the arrangement of questions in a survey may be more or less critical depending upon the particular population under study. Without prior hypotheses, the variations in the strength of the order effects were examined as a function of the respondents' sex and education. The effects of income were not examined because of its correlation with education, r = .41, and race was omitted because the small sample of nonwhites was not sufficient for partitioning. Duncan and Schuman (1977) have suggested that changes in question order might alter the patterns of relationship between the responses given to questions, as well as altering the frequencies of This content downloaded from 157.55.39.96 on Sat, 03 Sep 2016 05:49:34 UTC All use subject to http://about.jstor.org/terms 210 SAM G. McFARLAND particular answers. In the present study, for example, the strengths of the correlations between answers to the general and specific questions on particular topics might be either enhanced or diminished by placing the specific questions first. With no prior logical or empirical basis for predicting which, if either, would occur, the variations in the strength of the relationships between the specific and general questions as a function of question order were examined without advance hypothe-