Examination of Test Characteristics’ Effect on Coefficient α and Coefficient ω

Abstract

In this study, five factors were simulated to determine their effect on three measures of reliability: coefficient α, coefficient ω, and the true scale reliability as defined in a classical test theory context as the ratio of true score variance over observed score variance. The factors were the number of items, the level of item discrimination, the number of dimensions, the correlations among dimensions, and the location of the items in relationship to the latent ability score distribution. In all higher-order dimensional conditions, simple structure was assumed. The data were generated using the multidimensional item response theory compensatory two-parameter logistic model. As expected, when the number of items, the magnitude of the item discriminations, and the correlations among the dimensions increased, the reliability correspondingly increased. Noticeable differences were observed across all higher dimensionality conditions with ω values being significantly lower than α, a finding which could have been an artifact of the simulated simple structure.