Estimating a unidimensional latent ability across multiple test forms with confounding difficulty and length within sub-content areas

Abstract

This study investigated the effects of estimating unidimensional latent abilities for subgroups of a population across multiple test forms with confounding difficulty and number of items within sub-content areas. Examinees were grouped based on their true two-dimensional ability; estimates within subgroups were compared across test forms having equal length and average item difficulty overall, but differing numbers of items and/or difficulty within subsets. Examinees with differing true abilities across two dimensions had significantly different estimated scores across forms, depending on the alignment of true ability in each dimension with the number of items and average item difficulty of items within that dimension. These effects tended to decrease as the correlation between dimensions increased. The results of this study bring awareness to test developers for the need of controlling item specifications within sub-content areas.