The Potential Impact of Not Being Able to Create Parallel Tests on Expected Classification Accuracy

Abstract

In many practical testing situations, alternate test forms from the same testing program are not strictly parallel to each other and instead the test forms exhibit small psychometric differences. This article investigates the potential practical impact that these small psychometric differences can have on expected classification accuracy. Ten different sets of tests were assembled by minimizing the differences in test information at five θ locations. The impact of the psychometric differences between the assembled test forms was quantified for two different groups of simulated examinees across a range of possible cut scores. Results indicated that using sequential or simultaneous test assembly is preferred to random test assembly. Analyses also implied that the small differences in the psychometric properties between tests produced differences in overall classification accuracy that were less than 1.5%. The biggest differences in classification accuracy were found when the test information functions were not as well matched in regions where there were more examinees. Although these differences were fairly small, they may have the potential to have a practically significant impact on decision making. It is suggested that classification accuracy critically depends on the differences in test information, the location of the cut score, and the groups of examinees considered.