Evidence for Context Switching in the Effects of Average Item Length and Item-Length Variability on Internal Consistency

Abstract

ABSTRACTThis investigation examined the effects of 3 item characteristics—the average number of words per item, within-scale variability in item length, and item “direction”—on internal consistency reliability and interitem correlation. In Study 1, we examined the effects of these variables on overall scale-level reliability using 444 subscales from 9 personality scales. In Study 2, we examined interitem correlation at the paired-item level using 477 nonredundant item pairs from 14 personality scales. Lower scale reliability was associated with more average words per item, greater within-scale variability in item length, and a greater percentage of reverse-keyed items. Similarly, smaller interitem correlations were associated with a greater degree of mismatch in item length between the paired items and with a mismatch (vs. match) in the items' respective “directions.” The pattern of results across both studies supports our notion that lower internal consistency results from increased context switching; that is, from the confusion that occurs when respondents must switch back and forth between the interpretive frames pertaining to short versus long items, or between items pertaining to one pole of a personality dimension and its “opposite” pole. Suggestions for maximizing the internal consistency of personality scales are proposed.