DIMENSIONS OF RESPONSE CONSISTENCY IN PAIRED COMPARISONS

Abstract

ABSTRACTPeople are frequently called upon to make choices between objects on such grounds as value, pleasantness, etc. Whenever people make a number of such judgments, inconsistencies of choice are observed. Not only do different people make different choices, but some individuals are inconsistent with themselves. Questions may be raised whether such self‐inconsistency is a general trait. Is a person relatively inconsistent when compared with other people in all situations, or is his inconsistency so highly specific that no valid predictions can be made from one occasion to another? Or does self‐inconsistency depend upon the types of judgments made and the kinds of objects being judged?The present study explores the structure of individual differences in self‐inconsistency using as the basic measure of inconsistency the number of circular triads made in choosing between stimulus pairs presented in paired comparison schedules. Twelve paired comparison schedules were constructed–six called for judgments of similarity, six for judgments of preference. Three different kinds of stimuli were used in the schedules–colors, verbal material, and designs or figures. It was hypothesized that factor analyses of circular triad scores obtained for the 12 schedules would yield a general inconsistency factor and, possibly, additional factors reflecting the types of judgment and the kinds of stimuli involved.The schedules, along with selected aptitude and personality measures, were administered to approximately 370 high school juniors in three schools. Six of the schedules were administered at one school, six at another, and all 12 at the third school. The six measures initially administered at the first two schools were readministered a week later to the same subjects to obtain estimates of the test‐retest reliabilities of the inconsistency measures. Comparisons of the inconsistency score distributions were made between the two administrations and among the three schools.The principal findings may be summarized as follows:1. Strong correlational evidence was found that subjects who were inconsistent in making judgments of one set of stimulus pairs tended to be inconsistent in judging other sets of stimulus pairs, regardless of the type of judgment and kind of stimulus involved. All factor analyses conducted yielded general inconsistency factors upon which all the inconsistency measures had positive loadings.2. There was no definitive evidence that either type of judgment or stimulus differentially affected inconsistency scores.3. Measures of aptitude and ability generally correlated negatively with inconsistency scores except for measures on which males did comparatively well. (Males had significantly higher inconsistency scores than females.)4. The personality variables used in the study did not in general correlate significantly with inconsistency scores.5. Significant differences, which seemed to stem from differences in motivation of the subjects, were obtained in the means and variances of inconsistency scores across the experimental groups.6. Reliabilities of the inconsistency scores were low. The low reliabilities seemed to be related to the low variance and positively skewed distributions of the circular triad scores.From these findings, three principal conclusions were drawn:1. For subjects of the type used in this experiment, inconsistency in judging paired comparison schedules is to a considerable degree, at least, a general trait.2. The motivation of the subjects appears to have an important influence on the amount and pattern of inconsistency and must be taken into consideration in administering paired comparison schedules and in evaluating resultant circular triad scores.3. Longer schedules, using carefully selected stimuli, should be developed to increase the sensitivity and reliability of any inconsistency measures to be used operationally or in further research.