Latent Trait Modeling with the Common beliefs Survey III: Using Item Response Theory to Evaluate an Irrational Beliefs Inventory

Abstract

Assessment of irrational beliefs by such measures as the Common Beliefs Survey III (CBS) has traditionally relied upon classical test theory assumptions, in which the properties of specific test items are less important than the total test score as the aggregate of all item responses. An alternative approach using item response theory (IRT) methodology allows one to specify the parameters of difficulty and discrimination for each test item. Difficulty levels of CBS items range along a continuum of irrationality, the implied latent trait measured by responses to the questionnaire as a whole. We evaluated the CBS responses of 605 individuals from clinical and college settings, drawing from current and archival data. The original Likert scale ratings were recoded into dichotomous scores. Fourteen of the 54 items were highly or very highly discriminating in distinguishing respondents with high and low irrationality levels. However, discriminating items exhibited a very narrow range of difficulty; most functioned at a point a little above the halfway mark on the continuum of irrationality. Item characteristic curves and test information curves were very similar for female (n = 424) and male (n = 179) respondents. We derived a 4-item screening test for irrationality from our IRT analyses of the 54 CBS items. Further test development, focused on the selection and scaling of items with a much broader range of difficulty, would facilitate evaluation of the hierarchical structure of irrational beliefs.