The effect of item pool and selection algorithms on computerized classification testing (CCT) performance

Abstract

The purpose of this research was to evaluate the effect of item pool and selection algorithms on computerized classification testing (CCT) performance in terms of some classification evaluation metrics. For this purpose, 1000 examinees’ response patterns using the R package were generated and eight item pools with 150, 300, 450, and 600 items having different distributions were formed. A total of 100 iterations were performed for each research condition. The results indicated that average classification accuracy (ACA) was partially lower, but average test length (ATL) was higher in item pools having a broad distribution. It was determined that the observed differences were more apparent in the item pool with 150 items, and that item selection methods gave similar results in terms of ACA and ATL. The Sympson-Hetter method indicated advantages in terms of test efficiency, while the item eligibility method offered an improvement in terms of item exposure control. The modified multinomial model, on the other hand, was more effective in terms of content balancing.