Reliability of Preference Ratings as a Function of Cardinal Value and Natural Order

Abstract

Preference ratings generally involve a scale which employs the normal order of the natural numbers. The objective of the experiment reported here was to determine the relative reliabilities of preference ratings administered under the following 3 conditions: ( a ) when a normal order scale is used twice, ( b ) when a normal order scale is followed by a complete reversal of that order, and (c ) when a reversed order scale is used twice. Three groups of 20 Ss were asked to give their preferences for the following 9 fruits: bananas, oranges, apples, pears, plums, pineapples, watermelons, grapes, and peaches, by rank. Preference ratings on a reversed scale were translated back into the normal order, under conditions ( b ) and ( c ) , so that 9 which indicated maximum preference and 1 minimum preference, became 1 and 9, respectively, after translation. In short, the normal order scale value corresponding to a reversed scale value, SR, was equal to 10 SH. When 2 normal-order scale preferences had been obtained for each S, rho was calculated and converted to r by the formula, r = 2 sin ( x p / 6 ) . The average values of T for each group, namely, rl = .91, r2 = .67, and r3 = .77, were obtained, using the Fisher transformation, where n = 9 (our scale ratings) and is averaged for 20 values of z. A t test of significance showed that for and &, p < .01; while for i z , .O1 < p < .02. The significance of the difference between pairs of these ss was also determined, where the sample n = 20. Only and r2 were significantly different ( p < .05). Clearly the relative reliabilities of the 3 scaling procedures are in the order: normal, reversed, and mixed. Normal and mixed tend to yield credibly different results. No guess can be made as to what the results would have been for a larger N, scale values in excess of 9, for other stimulus materials in the taste category or for preferences in some other category.