Rating Scales: Numeric Values May Change the Meaning of Scale Labels

Abstract

Three experiments indicate that the numeric values provided as part of a rating scale may influence respondents' interpretation of the endpoint labels. In experiment 1, a representative sample of German adults rated their success in life along an 11-point rating scale, with the endpoints labeled at all successful and extremely successful. When the numeric values ranged from 0 (not at all successful) to 10 (extremely successful), 34 percent of the respondents endorsed values between 0 and 5. However, only 13 percent endorsed formally equivalent values between 5 and 0, when the scale ranged from -5 (not at all successful) to +5 (extremely successful). Experiment 2 provided an extended conceptual replication of this finding, and experiment demonstrates that recipients of a respondent's report draw different inferences from formally equivalent but numerically different values. In combination, the findings indicate that respondents use the numeric values to disambiguate the meaning of scale labels, resulting in different interpretations and, accordingly, different subjective scale anchors. NORBERT SCHWARZ iS program director at ZUMA and Privatdozent of psychology at the University of Heidelberg. BARBEL KNAUPER is research associate at ZUMA. HANS-J. HIPPLER iS project director at ZUMA. ELISABETH NOELLE-NEUMANN is founder and director of the Institute fur Demoskopie Allensbach, and professor emeritus of mass communication at the University of Mainz. LESLIE CLARK iS assistant professor of psychology at Purdue University. The reported research was supported by grant SWFOO44-6 from the Bundesminister fur Forschung und Technologie of the Federal Republic of Germany to Norbert Schwarz. We thank George Bishop, Tory Higgins, and Tracy Wellens for stimulating discussions. Correspondence should be addressed to Norbert Schwarz, ZUMA, P.O. Box 12 21 55, D-6800 Mannheim, Germany. Public Opinion Quarterly Volume 55 570-582 ? 1991 by the American Association for Public Opinion Research All rights reserved 0033-362X/91/5504-0008$02.50 This content downloaded from 40.77.167.53 on Fri, 09 Sep 2016 04:16:31 UTC All use subject to http://about.jstor.org/terms Changes in the Meaning of Scale Labels 571 Rating scales with labeled endpoints are probably the most widely used measurement instrument in social and psychological research. Leaving some concerns about their psychometric properties aside (see Nunnally 1978), the use of these scales does not seem to be very controversial (see Dawes and Smith [1985] for a careful discussion of their general properties and for empirical and psychological justifications for their use). In general, 7-point scales seem to be best in terms of reliability, percentage of undecided respondents, and respondents' ability to discriminate between the scale values (e.g., Cox 1980). Thus, seven plus or minus two is the usual recommendation. Moreover, respondents are able to use rating scales consistently, even in telephone interviews without visual aids (e.g., Hormuth and Bruckner 1985). In addition, verbal rating scales, which provide a label for each scale point, have been found to be more reliable than scales that provide labels for the endpoints only (Krosnick and Berent 1990). Finally, researchers have observed that the terms used to label the endpoints, or to designate the separate values of verbal rating scales, may affect the obtained distribution (e.g., Rohrmann 1978; Wegner, Faulbaum, and Maag 1982; Wildt and Mazis 1978). This suggests that respondents pay close attention to the meaning of the labels provided to them, much as one would hope. Whereas the number of scale points, the inclusion or omission of a neutral point, and the choice of scale labels have received considerable attention in the literature (see Dawes and Smith 1985), the specific numeric values provided have, to our knowledge, not been the topic of theoretical analysis and empirical investigation. Apparently, researchers assume that, for example, a 7-point scale that ranges from 1 to 7 is equivalent to a 7-point scale that ranges from 3 to + 3, as long as the same endpoint labels are provided. In the present article, we will question this assumption. Drawing on survey data from the Allensbach archive, we will first demonstrate that the specific numeric values provided in a rating scale can have a dramatic impact on the obtained results. We will then discuss different underlying processes and will test their viability in laboratory experiments. Experiments 1 and 2: The Impact of Numeric Values